It depends directly on the language and if the diacritic produces a new letter or simply a variation of the same letter.
In French (or Italian, Catalan, Portuguese...), accented characters (such as À, É, Ê, Ô, Ö, etc.) doesn't produce a new letter, they are only variation of the same letter. As such, one would expect words starting with an accented character to be categorised just as any other word that starts with the same non accented character.
In German, the umlaut produces three different vowels (Ä, Ö and Ü) that may be considered different letters but when alphabetically sorting words, the umlaut is usually not distinguished from the underlying vowel.
In Finish, Norwegian and Danish however, characters such as Å, Ö are actually completely different letters from A and O. In those languages, one will usually expect words starting with those characters to have their own category.
There are other languages such as Hungarian, where accented characters (such as á, é, ó, ú, ő, ű...) are different letters but are generally collated in pairs (a/á, e/é, i/í, o/ó, ö/ő, u/ú and ü/ű) in dictionaries.
As you can see, the same character (Ö for instance) may be treated differently depending on the language, you'll have to adapt the behaviour of your UI depending on the word's language, not only on the character itself.
You should check the wikipedia page about diacritics where you'll find if an accented character is considered a new letter in the languages you'll have to support :
- If for a specific language, the accented characters are not different letters, words should be in the same category.
- If you have to handle languages where diacritic generates new letters, you should check for each language and use the local common practice.
If you have to support multiple languages with different rules, you should delegate the i18n to a dedicated solution since as you can see, it can get very complex very quickly.