What is the best user testing method to validate proposed accelerators or system shortcut concepts?

Question

In the context of designing an interface for an internal application that aims to balance the needs of new and experienced users, we are looking to introduce accelerators, or system shortcuts, to enhance efficiency for those who are familiar with the system. We recognize the need to make these accelerators readily available, yet easy to ignore for those still learning the basics. Common practices, such as displaying keyboard shortcuts beside corresponding menu labels and using visual styles that don't compete with the label itself, have been considered.

Now we want to validate and refine these shortcut concepts through user testing.

What would be the most effective research method to test these proposed accelerators with users?

• How can we ensure that we're gathering insights that not only demonstrate their usability and learnability but also balance the needs of novice and experienced users?
• Additionally, how can we evaluate the impact of these accelerators on the overall user experience, including factors like efficiency, satisfaction, and potential cognitive load?

Answer 1

Without a lab setup and a very scientific methodology approached it's quite difficult to research this. Another thing that you could do is do some exploratory research with your users to learn more about what software do they use day to day, if they use keyboard shortcuts, what are the most used shortcuts that they can recall, why they use it, how did they learn it, how they approach a new software that has shortcuts, do they use cheatsheets or read the documentation or has the software they use a way to nudge, do they use software with no shortcuts? Do they feel the lack of shortcuts there is a major downside and then ask them to score shortcuts on their perceived:

• efficiency, satisfaction, speed, ability to learn etc.

That would be a first part on generative. Second part implying evaluation is difficult to achieve as you won't have users experienced with using shortcuts in your product, if you would you could have split users in two groups, using and not using shortcuts and hand them assignments and measure for speed of execution, error avoidance and then finally have them evaluate their perceived efficiency, satisfaction, speed etc.

Here are two pieces of research done on this topic: https://pure.tue.nl/ws/portalfiles/portal/47032129/630263-1.pdf https://www.researchgate.net/publication/249955485_Keyboard_Shortcut_Users_They_Are_Faster_at_More_than_Just_Typing