I have n-items (n being somewhere between 5 and 9) that I would like participants to sort in relative order of importance.

Although I have used visual 'drag and drop' approaches before, I found that this is quite hard to do with 9 items. People need to compare each one to all others etc.

There is a way that a perfect result can be achieved by asking a set of simple comparisons such as asking: which is better A or B, followed by 'which is better' C or A. etc.

I'm sure this has a name in the UX / Marketing business but my best efforts cannot seem to find it. With the name, I can look at previous work and best implementation approaches...

I'm sure it has the word 'Matrix' in it as it sort of should like like this

   A B C D E F
A  - < > < > <
B  * - < > < >
C  * * - < > <
D  * * * - < >
E  * * * * - <
F  * * * * * *

- : not possible
* : already answered so would not be posed
< > Row more/less important the column 

total questions would be


I would call it "pairwise comparison".

I recall that IBM Rational Focal Point uses this method to rank requirements.

The purpose is to get a list ordered by priority, even though the list of requirements could be overwhelming. Divide and conquer. The items of pairs that you are not able to answer still get a place in the list based on the result of those items comparison in other pair combinations.

  • Pairwise comparison is the term for the methodology described in the question. As mentioned, the number of comparisons gets out of hand quickly. Not all comparisons are necessary. Consider reviewing the conjoint analysis methodologies to identify a strategy for choosing the smallest subset of comparisons that provide the maximum amount of data. – user1757436 Jan 6 '13 at 20:15
  • This is great - i hadn't considered doing a 'close' enough ranking by doing this. It's sort of a 'quicksort' approach. Conjoint was the word that really connected – hobbit_be Jan 6 '13 at 20:34

I don't think there is a common UX term for a comparison method like this because in almost all cases it would be bad UX practice. Using a drag-and-drop method (or even just a method of clicking items one at a time) takes only 9 actions by the the user for 9 items, but this new method you suggest would take 36 user actions. Unless it's absolutely essential for the user to individually compare every item to get an exact order, I don't think this method is likely to be useful.

Answering your implicit question: you said that you've already tried a drag-and-drop, and it looks like you're searching for an alternative method of sorting items. Have you considered trying something like the following?

Initially, all items appear as buttons.


download bmml source – Wireframes created with Balsamiq Mockups

The user can then click these items one at a time to move them into the sorted column.


download bmml source

Once there are multiple items in the sorted column, the user has the option of changing his/her mind and shifting the order by clicking on the arrows on the right of each button.


download bmml source

This would be much faster and still give the user the option to change after initial comparison.

  • Ranking items, as suggested in this answer, will suffice if knowing the distance between items of adjacent ranks is not important. If the distance between the items is important then comparisons between pairs or triplets of items will provide data for calculating the distance. – user1757436 Jan 6 '13 at 20:24
  • Also i appreciate the drag and drop is very useful and work well with small and quick to judge items, as your excellent examples (cudos for the balsamiq links!). My data types are a whole paragraph and subtle and it is very streneous to actually do this accurately with the whole dataset. Also see this article. Also as mention a good implementation would 'only' :) need 15 Yes/No to get a perfect list. For full disclosure our list is 30+ items and we do a quick 'cull' as you did as well – hobbit_be Jan 6 '13 at 20:41

You could likely stick with the term Comparison Sort which is the overarching term for all the various sorting algorithms.

enter image description here

This type of sorting through a step process could be very effective in:

  • Enforcing full user consideration for maximum quality sort
  • Helping the user achieve the sort in the fewest number of mental considerations (they will likely not be doing a QuickSort via drag-and-drop)
  • Offloading the complexity to the system when the number of items is over the Magic Number 7 (The limits on our capacity for processing information which interestingly is disclaimed as +/-2 which is 5-9 as in your question)
  • great as this is from the perspective from computer science. And extra cudos the magic number 7 link - sort of discovered it in person. – hobbit_be Jan 9 '13 at 17:16

A/B testing or multivariate testing?

  • I don't think quite apply as my needed input is an individuals approach. However the multivariate is a interesting link and much appreciated – hobbit_be Jan 6 '13 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.