3

I'm doing a Tree Test on a product that runs embedded software. I am faced with a dilemma on how to design the tree and where to have the designated "correct response" be. I am using Optimal Workshop (Tree Jack).

We have constructed this information architecture based on the results of a previous card sorting study. I should note that users of the tree test may or not be familiar with the product and may not be familiar with the newly added features in this tree test. Additionally, users typically receive training prior to using the product and will receive no training on the tree test.

What is the recommended best-practice?

Choices are as follows:

1) Have the "correct response" in the tree be the actual item in question.

Example: Find where you can change your alert volume.

Device Settings > Volume > Alerts

During normal system use the user would then be able to make adjustments to this setting after opening the menu.

The pros of this choice seem to be that it will be able to simulate how a user may search through the navigation on the embedded product in that if they access the wrong menu they would then go back. Through the results we would be able to see if they navigate down a wrong menu before eventually finding the correct answer.

2) Have the "correct response" be the menu which contains the feature/item.

Example: Find you where you can change your alert volume.

Device Settings > Volume

The pro of this seems to be that it will put more emphasis on testing if the label clearly communicates what it contains, choosing the correct label, and if it contains what user's believe it contains.

Improve this question
2
  • One other thought: In a way these two things would be testing the same things since the results would show if they don't take the optimal path but with Option #2 I would be able to collect subjective confidence scores to indicate how strongly people feel an item should be under a menu.
    – Arctic
    Feb 14, 2017 at 2:08
  • Related, but not an answer: Make sure to phrase your questions in ways that don't lead subjects to the answers. For instance, "Find where you can change your alert volume" leads people to look for "alerts" and "volume." Instead, try "The notifications the system sends you are too loud. Where would you go to adjust them?"
    – Ken Mohnkern
    May 15, 2017 at 17:49

1 Answer 1

Reset to default
3

End on the layer you want to test the efficacy of

You can't really make any definitive statements about the layers below.

How you define success depends on how you phrase the question.

If the question/scenario is to find where, specifically, alert volume is changed, simply navigating to the layer above doesn't say anything definitive about the label 'Alerts'. It may be that the label for the Alerts item is ambiguous or obscure or in a long list.

If your test ends at the 'Volume' layer, it would be incorrect to say that users can find where to change alert volume.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.