In a word: dysiconia. That's my own coinage, by analogy to dyslexia, and means that I have a really, really, really hard time trying to figure out what the stupid little pictures in a typical GUI are supposed to mean. I know they're supposed to be intuitive, but to me they're not. I think this might be true of more people that you'd suspect, or why else do GUIs often add text below their icons?
Of course with extended use I would memorize a few common icons (though I still can't understand why they're supposed to be 'intuitive'), but confront me with a new one and once again I'm clueless. Worse, there's no good way to look up a picturepicture**. With an unfamiliar command, I can simply run "man strange_command", look in the index of the program manual, or use a Google search.
Humans are naturally verbal creatures - after all, language is what's supposed to distinguish us from the rest of the animals, isn't it? For those of us who use alphabetic languages, a good reading vocabulary in our native language would be upwards of 50,000 words. (Those with multiple languages might easily add 10K or more per language, depending on fluency.) We can either look up unfamiliar words in a dictionary, or deduce their meaning from context. Adding the few thousand (not hundreds of thousands) needed to use a CLI to this base is easily done.
All this is to say that for most literate* people, using a terminal/CLI is just an easy extension to skills they already have. Once one becomes familiar with with it, it is generally superior to graphical methods for any task that isn't inherently graphical (such as drawing).
PS for @nekomatic: For evidence, how about this: http://www.pewresearch.org/fact-tank/2015/10/19/slightly-fewer-americans-are-reading-print-books-new-survey-finds/ Literally the first link found searching for "percent of americans who did not read a book this year". From the link "Some 27% of adults said they hadn’t read any books over the past year..." and "The median number of books read by women was five, compared with a median of three for men..." If that's not functional illiteracy, what is?
*I suspect the popularity of GUIs goes hand-in-hand with the increase in functional illiteracy. Likewise voice input & output.
**Even ideographic languages like Japanese (and I think Chinese, though I've never studied it) have systems for looking up kanji in a dictionary.