20

We have a list of countries in our UI where the design prototype groups the countries by initial letter. The country names change by language, so the grouping also changes. This also means that the groups themselves change.

In some languages, country names start with accented characters, e.g Austria in English, and Österreich in German. How is it more common to group these words: should Österreich belong to the "O" group or to the "Ö" group? Are there different traditions in different countries (I often hear Germans refer to "a" and "ä" as different letters)?

Another example in Portuguese: Alemanha and Áustria both start with A and no one would expect them to be in separate groups.

Improve this question
5
  • Is it a standard "select your country" -list, or does the user select multiple countries?
    – Boat
    Feb 6, 2018 at 12:22
  • 4
    Just assume it's a list of grouped labels. The fact that they are countries is not relevant in this case.
    – ecc
    Feb 6, 2018 at 13:17
  • 6
    Is there any way you could do this in a data driven way? That is, sort and collate the string using a built in locale-aware sorting function. Then generate the groupings based on the actual letters you have data for. This is still a problem for languages like Finnish where "W" and "V" are treated as the same letter and don't have a sort order between them. It would make your life 100x easier if you can let your OS vendor keep track of all that i18n stuff.
    – Ukko
    Feb 6, 2018 at 16:06
  • 3
    This is what locales and collations are for. Set these correctly and your programming language's sort function should take care of the rest.
    – Sammitch
    Feb 6, 2018 at 19:00
  • Can you list them twice?
    – Kevin
    Feb 6, 2018 at 20:37

7 Answers 7

Reset to default
36

It depends directly on the language and if the diacritic produces a new letter or simply a variation of the same letter.

In French (or Italian, Catalan, Portuguese...), accented characters (such as À, É, Ê, Ô, Ö, etc.) doesn't produce a new letter, they are only variation of the same letter. As such, one would expect words starting with an accented character to be categorised just as any other word that starts with the same non accented character.

In German, the umlaut produces three different vowels (Ä, Ö and Ü) that may be considered different letters but when alphabetically sorting words, the umlaut is usually not distinguished from the underlying vowel.

In Finish, Norwegian and Danish however, characters such as Å, Ö are actually completely different letters from A and O. In those languages, one will usually expect words starting with those characters to have their own category.

There are other languages such as Hungarian, where accented characters (such as á, é, ó, ú, ő, ű...) are different letters but are generally collated in pairs (a/á, e/é, i/í, o/ó, ö/ő, u/ú and ü/ű) in dictionaries.

As you can see, the same character (Ö for instance) may be treated differently depending on the language, you'll have to adapt the behaviour of your UI depending on the word's language, not only on the character itself.

You should check the wikipedia page about diacritics where you'll find if an accented character is considered a new letter in the languages you'll have to support :

  • If for a specific language, the accented characters are not different letters, words should be in the same category.
  • If you have to handle languages where diacritic generates new letters, you should check for each language and use the local common practice.

If you have to support multiple languages with different rules, you should delegate the i18n to a dedicated solution since as you can see, it can get very complex very quickly.

Improve this answer
12
  • 2
    To continue the list: in Hungarian, A/Á are different sounds (not just different length) but sorted together, same for E/É. Then the I/Í, O/Ó, Ö/Ő, U/Ú and Ü/Ű pairs only differ in length and are sorted together. All of these are considered "different letters" (so our alphabet has 14 vowels).
    – molnarm
    Feb 6, 2018 at 16:01
  • @MártonMolnár so you're saying that in Hungarian, accented characters are different letters (which is in line with the linked Wikipedia page about diacritics) but a Hungarian would expect words to be sorted together ?
    – zakinster
    Feb 6, 2018 at 16:06
  • 2
    Yes, the article is correct. (Sorting in Hungarian is actually even more complicated, so some software, e.g. SQL Server use a "Hungarian_Technical" collation to avoid some of these issues.)
    – molnarm
    Feb 6, 2018 at 16:14
  • 1
    @StarWeaver In French at least, the accent is completely ignored (all E would be treated the same) in the sorting order and it would be the following letters that would take precedence to define the order of the words. If the only difference is the accent, I’m not aware of any general rules, but this case would be rare.
    – zakinster
    Feb 6, 2018 at 19:22
  • 1
    @O.R.Mapper I updated the answer accordingly
    – zakinster
    Feb 7, 2018 at 8:14
5

As a German, I know of two ways how this is handled. Either, "ö" is treated as "o" (e.g., in an encyclopedia) or it is treated as "oe" (e.g., in a phone book). "ß" is always treated as "ss".

Even the German ISO (DIN) knows these two variants: DIN 5007 Variants 1 and 2

On the software side, database like MySQL have different collations.

There is no definite answer on this. Maybe this table helps (no, honestly, it probably makes things worse).

Improve this answer
4

For a danish site we faced the same issue, and we grouped them as separate characters to avoid the confusion. The grouping needs to be understandable for the users.

Gyldendal.dk authors list

Improve this answer
2
  • 14
    This may be correct for Danish characters (which may be considered completely different letters as I understand). But I don't think this approach would work in a language where the accentuated characters are only a variation of the same character (e.g in French, the word Élément would be expected in the E section), what about the German examples in the question ?
    – zakinster
    Feb 6, 2018 at 15:12
  • @zakinster you are right, this example is for Danish characters and that's what I said. "The grouping needs to be understandable for the users." In other languages if it makes sense to group accentuated characters with regular ones, then it should be done. It can vary language to language.
    – Usman Mani
    Feb 7, 2018 at 6:32
4

The rule is that you should put names into categories, and sort them, in such a way that your users will find them where they expect them. You follow the rules of the user's language. For example, if you have a Swedish name like Ångström, and your users are British, you sort it under the letter A because that's where British users would look for it. If your users are Swedish, you sort it after the letter Z.

Forget about figuring out the correct rules yourself. They are complicated, even worse when you don't know the language (like in German, you'd need to know which one of two completely different systems to apply). Check out if your operating system has any useful libraries to suppotrt you.

Improve this answer
5
  • 1
    Unfortunately, common "operating system libraries" tend to assume "one language = one sorting", ignoring issues like the "two completely different systems" you mention. At least, that's my impression from common runtime libraries that provide some locale support.
    – O. R. Mapper
    Feb 7, 2018 at 6:45
  • @O.R.Mapper - Really? The C++ Standard Library has std::collate; an example from the docs that are installed here is std::sort(v.begin(), v.end(), std::locale("sv_SE.UTF-8"));. There are also libraries such as ICU which provide additional support if you need more than what's in the standard library.
    – Toby Speight
    Feb 8, 2018 at 11:40
  • 1
    @TobySpeight: Your standard library example appears to confirm exactly what I said (sorting order exclusively coupled to language/locale identifier, without any chance to distinguish between different sorting orders based upon factors that are orthogonal to the language). Am I misreading anything there? The ICU library does, in contrast, seem to support different sortings per language, though (and seems to me like a specific effort to make up for the deficiencies of common "operating system libraries" in that regard).
    – O. R. Mapper
    Feb 8, 2018 at 11:47
  • Oh, sorry - I misunderstood you. I was thinking of environments where you need to to a (process-wide) setlocale() each time you wanted to use a different language. But to answer your question - if your platform provides (or you are willing to write) collation rules for your language-variant, you can use them in the standard library. The name is an opaque identifier - it's only convention that it starts with the ISO 639 code.
    – Toby Speight
    Feb 8, 2018 at 11:55
  • I haven't checked, but I suspect that collation might be part of the difference between ca_ES and ca_ES@valencia on this (Debian) system, and it's certainly a factor in different countries' variants of some languages (such as European and American versions of Spanish).
    – Toby Speight
    Feb 8, 2018 at 12:00
2

Another way of stating what others said is that it depends on the collation (text sorting) algorithm used. Each language/dialect/language-variation has its own collation algorithm [I have a vague recollection that some language(s) has(ve) more than one?]. Some collations consider letters with and without diacritics to be the same letter (so e.g. xaxx, xáxy, xayy). Some collations consider them different letters.

So a complete solution would have to "open up" the collation rules and look inside.

A quick-and-dirty solution might be to sort the strings (country names) with a black-box collation algorithm (form your favourite programming language or library) and try to group them by first letter [*]. If some letter comes up mixed with some other letter (such as 'A', 'Á', 'A') then you group them as the same letter. Not foolproof. :-)

[*] Note that "first letter" has some subtleties. First Unicode code point? First glyph? First glyph cluster? Unicode has combining characters...

Update: By the way, perhaps you can change your UI slightly (?) to avoid the grouping problem. If the grouping is just a way to quickly find a given string, much like in a dictionary, then an alternative is to have all your strings (country names) in an alphabetically-sorted (paginated?) long list and select some headings and link each to the matching location on the list. Such headings could be a single letter or the first few letters of the first string starting with that letter(s). It may work well to select approximately evenly spaced headings.

E.g. for a certain list of countries, Afg... Bhu... Con... Fra... Hun... Lib... and so on, every 25 strings each.

Note: just an idea, not tested for usability, details left as exercise ;-)

Improve this answer
1

Ä, Ö, Õ, Å etc. are not accented letters, but completely different and different sounding letters than A and O, in Swedish and Finnish alphabet they are in the end of the "english" letters. I have seen some lists that were organized by mixing up O and Ö, and that is as confusing as mixing up, let's say s and k.

If you think only in alphabetical point of view, you should find a formal standard that includes non-english letters. From usability point of view, I don't think alphabetical affects much how easy to use the list is.

For example you don't necessarily know what keyword you should use for your own country: Holland vs. Nederlands vs. The Netherlands. This is a great article: https://www.smashingmagazine.com/2011/11/redesigning-the-country-selector/ that suggests searching the list and mapping multiple keywords for the same country.

Improve this answer
3
  • 7
    "are not accented letters, but completely different and different sounding letters" > The problem being that this definition is totally up to the country that uses them and asking around gave me different answers. Some people do expect Ö- to fall in the O box. I agree about your last paragraph but that's irrelevant to the question at hand.
    – ecc
    Feb 6, 2018 at 13:13
  • 5
    To further @ecc's comment: before 2010 Spanish considered ch and ll to be separate letters affecting alphabetic sorting. I'd put money on there being several other languages that have similar rules concerning compound letters, a great many more if you expand your scope beyond the latin alphabet. IIRC Spanish still considers ñ to be a separate letter though coming directly after n and being a single character means it has much more limited effect for sorting/grouping purposes.
    – David Spillett
    Feb 6, 2018 at 14:43
  • 1
    I think in the end each character will have to be handled separately for each language.
    – ecc
    Feb 6, 2018 at 15:00
1

Probably oversimplified, but I'd just check a dictionary in the particular language and follow whatever way they alphabetized. It will not fit into a nice neat box. Different languages, different rules, I suspect...

Improve this answer
4
  • 2
    "a dictionary in the particular language" - note that in some languages, there are several valid collations that are used in different contexts. E.g. in German, dictionaries tend to be sorted according to slightly different rules than phone books.
    – O. R. Mapper
    Feb 7, 2018 at 6:38
  • Agreed, different countries/languages would do things differently, but if you followed the rules of a dictionary for the particular/language, regardless of 'rules', you would probably end up sorting things in a way that people from that country/language were used to, no? I mean, I have no knowledge or experience on how they print dictionaries in Germany, but then, odds are, I never would. German people/speakers, however, would most likely be much more familiar with a German dictionary and how to look up words in it, so...
    – Owen Parker
    Feb 8, 2018 at 14:36
  • I'm not sure how you can "follow rules regardless of rules" :) But my point is simply that just because a dictionary in language X uses a particular sorting, one cannot always conclude that this is the sorting that would typically be used in language X for an arbitrary other situation.
    – O. R. Mapper
    Feb 8, 2018 at 14:43
  • The entire basis for my comment was that of all the ways you might find to sort something, the dictionary method would probably be the most widely known method by the people of country X. In the old days, the phonebook might run a close second. These days, the very word phonebook itself is too archaic to consider. People 'these days' don't really use any 'book/hard copy' to look up anything. Just use autocomplete with smart suggestions and be done with it :)
    – Owen Parker
    Feb 9, 2018 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.