The issue with this type of layout is that it works well where each letter has a lot of content associated with it. For smaller glossaries, I prefer to initially use a telephone keypad arrangement where letters are grouped, so "ABC", "DEF" and so on in tabs. This approach means that you have fewer potentially fiddly mouse clicks to make to access content.
If needed, this design can be developed into a more segmented approach once a critical mass of content is reached. I'd ensure in this individual letter approach to show which letters have associated content and which do not (simple styling will work for this).
However: Before doing any of this, I'd speak to users and understand what they currently use, how they feel it works, and test any approaches with them to get feedback.