What do you call the method for ranking a set of items using a series of questions comparing just 2 at each time

Question

I have n-items (n being somewhere between 5 and 9) that I would like participants to sort in relative order of importance.

Although I have used visual 'drag and drop' approaches before, I found that this is quite hard to do with 9 items. People need to compare each one to all others etc.

There is a way that a perfect result can be achieved by asking a set of simple comparisons such as asking: which is better A or B, followed by 'which is better' C or A. etc.

I'm sure this has a name in the UX / Marketing business but my best efforts cannot seem to find it. With the name, I can look at previous work and best implementation approaches...

I'm sure it has the word 'Matrix' in it as it sort of should like like this

   A B C D E F
A  - < > < > <
B  * - < > < >
C  * * - < > <
D  * * * - < >
E  * * * * - <
F  * * * * * *

- : not possible
* : already answered so would not be posed
< > Row more/less important the column 

total questions would be

  (n-1)n/2

Answer 1

I would call it "pairwise comparison".

I recall that IBM Rational Focal Point uses this method to rank requirements.

The purpose is to get a list ordered by priority, even though the list of requirements could be overwhelming. Divide and conquer. The items of pairs that you are not able to answer still get a place in the list based on the result of those items comparison in other pair combinations.

Answer 2

I don't think there is a common UX term for a comparison method like this because in almost all cases it would be bad UX practice. Using a drag-and-drop method (or even just a method of clicking items one at a time) takes only 9 actions by the the user for 9 items, but this new method you suggest would take 36 user actions. Unless it's absolutely essential for the user to individually compare every item to get an exact order, I don't think this method is likely to be useful.

Answering your implicit question: you said that you've already tried a drag-and-drop, and it looks like you're searching for an alternative method of sorting items. Have you considered trying something like the following?

Initially, all items appear as buttons.

^{download bmml source – Wireframes created with Balsamiq Mockups}

The user can then click these items one at a time to move them into the sorted column.

^{download bmml source}

Once there are multiple items in the sorted column, the user has the option of changing his/her mind and shifting the order by clicking on the arrows on the right of each button.

^{download bmml source}

This would be much faster and still give the user the option to change after initial comparison.

Answer 3

You could likely stick with the term Comparison Sort which is the overarching term for all the various sorting algorithms.

This type of sorting through a step process could be very effective in:

• Enforcing full user consideration for maximum quality sort
• Helping the user achieve the sort in the fewest number of mental considerations (they will likely not be doing a QuickSort via drag-and-drop)
• Offloading the complexity to the system when the number of items is over the Magic Number 7 (The limits on our capacity for processing information which interestingly is disclaimed as +/-2 which is 5-9 as in your question)

Answer 4

A/B testing or multivariate testing?