User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
We are improving our toolbar in our web app. We are planning a tree test or similar to test the tools out.
A bit stuck on how to properly test the hierarchy and order of it. Any available methods or strategies?
Any best practices from Adobe, Figma, or google doc?
It looks like you may want to set up a Card Sort if you intend to test the various groupings and subgroups for your tool bar. So what you should do is put together the entire list of features and functions, as well as the existing categories and groupings for comparison.
There are different types of card sorts you can perform as well as different tools that you can use, so I suggest reading up more about the method and see if it is something effective for the questions that you want to answer from the research.
In terms of best practice, I think there isn't any specific consensus on the grouping but there are obviously some general conventions that you can follow if it is from a similar type of application (e.g. Microsoft vs. Apple, web app vs. desktop app).
The toolbar you attached does not show a hierarchy that in my opinion could be tested properly in a tree testing setup. You grouped tools together but this is not a hierarchy, since the tools are all represented in the same level.
So I think a good way to test is a usability testing. Giving users a task an observe how they use the toolbar (do they have to search for the right tool? are some tools searched for in another group? etc.)
Besides of the order you should also test the visual representation of the tools. They are not labeled which might be fine in a lot of cases if the icons are very common but not if there are less common tools with less widely used icons in place.
This looks like a good candidate for timed task tests. You can either do these observationally (with a group of test participants) or naturally (using playback recordings in a live environment, if you have the ability to do a series of AB tests and record them with something like Hotjar).
You're probably looking at answering questions like "Is it better to display paragraph formatting before letter formatting?" rather than "Is it better to show bold before italic?" because users expect bold before italic in most of the software that they use. So, you could run an experiment by showing paragraph formatting before letter formatting to half your test group, and letter before paragraph to the other half, and finding out which is accessed quicker.
Participant testing
You would record the tasks and play back the recordings for the timing data (telling users they're being timed might make them nervous). You can get some self-reported ease-of-use metrics and attitudinal feedback as well by talking to users after each task ("On a scale of 1 to 10, how easy was this?" "How would you make this better, if you could?"). You could also dig into struggles if you see them. This can be done moderated (in person over Zoom) or unmoderated (using a tool like TryMyUI or Usertesting.com).
Pros: You could get some qualitative feedback along with quantitative timings. It might be easy to piggyback this on to another user test in the same system, since formatting text is something most people do when using computers.
Cons: You might need to incentivize the test, which could be expensive. And that might limit how many data points you can collect.
User Session Playback Recording Data
If you're looking at playback recordings, the playback software could help you find people who clicked these particular buttons, and you could look at the timings between their paragraph/word selection and clicking the buttons.
Pros: You can get many data points without having to pay anyone for their time
Cons: You can't get qualitative feedback or ask users their thoughts
