I am a web team of one at a community college. I've never done in-depth user research by myself before and did my first card-sorting activity today using the online tool Concept Codify. I have a hierarchical clustering analysis and similarity matrix. How do I read them?
1 Answer
I have to say that for a site dealing with UX, their usability is quite worrying.
But anyway, I'll start with the similarity matrix because it is quite easy to interpret.
Similarity matrix
The 1st row is the same as the 1st column, the 2nd row the same as 2nd column, and so forth.
The matrix simply shows what percentage of participants agreed that the two items should be in the same group.
So, for instance, in the matrix above, 100% of participants agreed that Reply to thread should be in the same group as Start a favorite song pool (2nd and 3rd items).
But only 77% agreed that Ask someone about an artist's latest album should be in the same group as Respond to favorite artist poll.
The diagram is produced so nearby items produce string clusters, this why you often see strong agreement around the diagonal:
But you should lookout for things like this:
It mean 60% of the participants have put Find new artist and Find new song in the same group, but when all items where taken into account, stronger clusters formed when the two where not in the same group (the maths behind it is rather complex, but just consider that if you have 60% agreement that A has to be with B, it still doesn't mean that any group with A and B will have higher agreement than, say, a group with [A, C, D, E] and [B, X, Y, Z].
From a UX perspective, it means that if someone has found A in group X, and then asked to find B, they are likely to navigate to group X again.
Just as a side note, you may find the whole matrix easier to understand if shown in this format (from the OptimalWorkshop site):
Hierarchical clustering (Dendrograms)
If I'm honest, I'm pretty sure the diagram provided is incorrect:
It shows that item 3 is split from item 1 and 2, but the similarity matrix shows 100% agreement between the three - so they should all split from the same point.
It should really be something more along these lines:
It is mostly a clearer visualisation of the similarity matrix. To find the agreement between any two items, you trace the lines coming out of them, until they meet. The further from the label, the less agreement.
So in the illustration above, participants didn't quite knew where to put bananas and tomatoes (somewhat obvious given the other items), but you can see two clear other clusters there.
These type of visualisation is quite useful when you come to design menus with big items or the structure of complex sites. You just need to decide which split will be a menu item, and you can fairly easily tweak the depth, breath and labels of the eventual hierarchy items.
Although somewhat technical, you can learn more on dendrograms here.
