How to display hash digests

Question

What textual format should I use for presenting hash digests, to best support human comparison of two hash values?

The standard format is a sequence of hex digits, but I'm not sure whether that is the best choice. The task I want to support is that some person will need to look at two hash values and confirm whether they match. I am especially focused on usability. What format will work best for average people, and maximize the likelihood that they check diligently, and make the comparison process as efficient and effective as possible?

I can imagine many possibilities: e.g., hex, decimal, base64, or a sequence of words from a Diceware/PGP/EFF-style wordlist. The hash value could be one long string of hex digits, or it could be broken into chunks. There are probably other options, too. Is there any evidence for what is most effective? Are there any published user studies on the subject?

Answer 1

ASCII art encoding is one option, like used in ssh where it is known as randomart. For example:

+--[ RSA 2048]----+
|       o=.       |
|    o  o++E      |
|   + . Ooo.      |
|    + O B..      |
|     = *S.       |
|      o          |
|                 |
|                 |
|                 |
+-----------------+

There is some evidence that humans can compare these better than other methods. In A Study of User-Friendly Hash Comparison Schemes (free pdf), Hsiao et al found that it had the best average accuracy both when making "easy" comparisons (completely different fingerprint) and "hard" comparisons (ones where the pictures were brute forced to be close).

An important caveat is that:

Hash comparison schemes provide notably less security than the underlying hash functions (20 to 30 bits of entropy versus 128 or more bits). Protocol designers must be aware of this limited entropy and design schemes that prevent an attacker from brute forcing the hash comparison scheme.

That basically rules out securely using (only) randomart on a business card, for example. You would need a full(er) hash representation as well to avoid someone brute forcing a collision.

---

As an aside, they also make an important point regarding word-based encodings like suggested in the other answer. Someone who is not a native English-speaker will harder time comparing English words. For that reason methods that are language-independent may be preferable.

Answer 2

Take for example the SHA-256 hash of a null string:

e3b0c442 98fc1c14 9afbf4c8 996fb924 27ae41e4 649b934c a495991b 7852b855

It is shown here with break spaces at 32-bit intervals, 8 sets of 8 characters. It is reasonably readable by a human, but it is a little long. A 512-bit hash is even longer:

cf83e135 7eefb8bd f1542850 d66d8007 d620e405 0b5715dc 83f4a921 d36ce9ce 47d0d13c 5d85f2b0 ff8318d2 877eec2f 63b931bd 47417a81 a538327a f927da3e

Still reasonably readable if you pay attention, though it may be hard to keep track of where you are reading if you are attempting to compare it to another block. Generally the hash you are comparing it against will either be the same or very different.

When comparing 2 identical hashes, you need to compare the entire hash to be sure. For PGPfone, a word list was developed to compare the authentication strings. This was designed primarily for comparison of a voice channel, each byte of comparison takes 1 word. The EFF word list is primarily designed for password use to replace the Diceware word list. You need 2 words from these lists to encode 3 bytes of the hash. While easier to compare than hex characters, a 256-bit hash would need 22 EFF words or 32 PGP words to display, taking up a lot of screen space, like this example:

reimburse pavilion starlight subtly jolliness unretired crimp peculiar polar partly ascent erased brisket small engaged cabdriver molehill alibi hardly reviving acid container

I honestly do not think that is much easier to compare than a hex string. For a visual comparison it makes sense to use the properties of human vision to reduce the amount of characters compared. We see colors (generally) with great accuracy, and can tell that red is different than blue without having to think. We can see alignment of visual information within a block. We can see if something is tilted or rotated slightly, or is offset from a common line.

You could choose 4096 words from the EFF list, and use 2 bits for color, 1 bit for caps/lower case, and 1 bit for bold+underline, and you could shorten the words required to 1 per byte pair.

Selecting the shortest words would make that more effective. I used 1 bit for alignment, 2 bits for color, and 5 bits for character to come up with an example of encoding bytes that requires a single character per byte. The data is then aligned in a 20 character wide grid, with 16 characters per row, and as many rows as required to display the hash. SHA-256 would need 2 rows.

For each byte, we encode the first 5 bits as a base 32 character, 0-9,A-Y, skipping IOS. The next 2 bits determines the color, 00=0xFF0000, 01=0x00FF00, 10=0x0000FF, 11=0x000000. Those colors are for computer display on a white background for someone who is not color blind, the colors can be changed for CMYK print, other backgrounds, etc. The final bit determines "alignment" within the grid. It can be a change in vertical or horizontal alignment within the grid squares, or a slight rotation, or bold print, font size, or some combination. The parity of the alignment bits within a 32-bit block determines the left or right alignment of the 4 bytes in a 5 space wide section of the row.

Using an offset of top aligned bold, e3b0c442 might be displayed as such: E3 = 11100 01 1 = char 28 "V", green, offset B0 = 10110 00 0 = char 22 "N", red C4 = 11000 10 0 = char 24 "Q", blue 42 = 01000 01 0 = char 8 "8", green

I will follow up with a complete hash example when I have the time, but the TLDR is to make use of the way we perceive visual information to both speed up the comparison while reducing the amount of pixes/area required to display the hash. Where nothing more than plain text is available, broken aligned hexidecimal or a high density wordlist are probably the best options. The algorithm used for SSH key fingerprint randomart is subject to collisions, fc94b0c1 e5b0987c 58439976 97ee9fb7 and fc94d0b8 21d9c84d 27439976 97ee9fb7 should produce the same visual fingerprint, for example.

Answer 3

I've found the shorter the string, the easier it is for humans to compare. As such, base 64 would be my recommendation for human readable binary comparisons.

On the other hand, if those humans need to understand the binary underneath the encoding, then hex is typically better, as it allows humans to visualise the binary in 4 bit increments, which (obviously) divide the 8 bit bytes quite neatly.

Answer 4

There is a substantial amount of research on this question in the literature. My summary of the literature is: the best textual representation is generated sentences.

In this scheme one deterministically generates a plausible-looking English sentence from the fingerprint and uses that to compare. An example sentence would be "The basket ends your right cat on his linen." -- semantically meaningless, but grammatical.

I am basing this conclusion on the following research papers, which ran user studies to compare multiple methods:

Can Unicorns Help Users Compare Crypto Key Fingerprints? Joshua Tan, Lujo Bauer, Joseph Bonneau, Lorrie Faith Cranor, Jeremy Thomas, Blase Ur, CHI 2017.

An Empirical Study of Textual Key-Fingerprint Representations. Sergej Dechand, Dominik Schürmann, Karoline Busse, Yasemin Acar, Sascha Fahl, Matthew Smith, Usenix Security 2017.

A Study of User-Friendly Hash Comparison Schemes. Hsu-Chun Hsiao, Yue-Hsun Lin, Ahren Studer, Cassandra Studer, King-Hang Wang, Hiroaki Kikuchi, Adrian Perrig, Hung-Min Sun, and Bo-Yin Yang, ACSAC 2009.

The Hsiao paper compared various methods and concluded that a Base32 representation was the best textual method they evaluated. However, they didn't test the sentence-based representation. The later two papers did a more thorough evaluation with more schemes, and both found that sentences performed significantly better than the other textual alternatives.

There are also graphical, non-textual schemes, which have some advantages and disadvantages. Tan et al. found that the best graphical schemes are more usable than textual representations, but also somewhat less secure: people make more mistakes. In particular, people sometimes accept two images as identical when they shouldn't, which can cause a security failure. This happens 10% of the time for the best graphical scheme, vs 6% of the time for sentences. So, graphical representations might be chosen if security needs are less important than usability. Sentences might be chosen if security needs are more important.

See the papers for more analysis, quantitative user study results, and advice for implementors.