Quantitative Analysis of Cardsort Data

Question

I'm working on some quantitative analysis procedures for cardsort (rank-ordering) type data. Are there established best practices for something like this? I was thinking cluster analysis, logit regression, or QCA as candidates.

Answer 1

Have you looked at the tools offered by Optimal Workshop in their OptimalSort and Treejack?

They offer a Similarity Matrix for identifying obvious clusters and dive into alternate pairings.

Dendrograms to visualize content groups and the top labels by participants.

Participant-Centric Analysis that produces different IA structures.

I found that rather than doing detailed analysis, you can first set up a card sort and then validate the groupings by looking at the task success rate in the tree analysis. It goes along with the concept of failing quick and fast to find the best IA for your content.

Answer 2

I have used Donna Spencer's fantastic card sort analysis spreadsheet and find it to be a handy tool. She provides instructions on how to use it, and it comes fully loaded with formulas and macros. It's much easier to use on a closed card sort where you have provided users with the category names. On an open card sort, you have to use your best judgment to normalize the user-provided names into common category names. And that is the most difficult part. Good luck! I find that there is still a lot of manual work involved in trying to figure out people's intent behind their name choices, but you will still have access to individual records, so you can spot patterns within their choices and apply your own boundless empathy to make sure you're not misinterpreting.