Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

After my company has recently been acquired, we have been allotted an hour of "usability testing" at an upcoming conference. The catch is, if we follow the template that has been put forth to us, it's not actually a usability study anymore, it's a focus group of 20 people in a room for an hour.

We will not be speaking directly with end users, but rather their managers etc. So while we will gain some great insight into how they envision the administration and setup of the software, so far it seems like a "tell us what you think/want" session, versus having the users show us how they use it, which is a cardinal sin in usability testing.

So far I'm thinking of opening with a 5-second test to gain initial impressions before we introduce any bias. From there, a quick demo (which is required in the session), and then perhaps having the group perform a task through consensus. I feel that this would help us see how they justify the actions and verbalize their thoughts as a group, but I am a little worried about an "alpha" participant skewing the results. And then closing with the participants explaining how this would fit into their existing system, and what features they are looking for.

Does anybody have any experience making a focus group more usability/user experience based? Any feedback or critique on my (rough) plan for guiding the session? I feel like I can't find any resources that combine the two.

I should add that while group usability testing would be great, we only have one screen (which is why I was contemplating navigation by consensus.

share|improve this question
2  
What a challenge! Could you try collect some "usability goals" from these folks instead? Rate of success, time per task, efficiency, effectiveness, etc, etc. Then use this as a basis for further "quality assurance". Letting them perform a task through consensus sounds like a good idea. Perhaps you could use the goals you have set to point out the areas that needs more attention and actual user-testing afterwards ;-) –  Jørn E. Angeltveit Feb 12 at 17:24

1 Answer 1

up vote 1 down vote accepted

If you can actually get the users to quickly perform specific tasks rather than doing a demo that would be be helpful as you can observe their reactions and user flow and define what are the usability concerns.

If that's not feasible, One approach you can do is to define what are the different user flows you want to evaluate (it would be good if you can ensure the number of user flows you have is equal to the number of the number of participants).

Once you have the user flows defined:

  • you could ask each participant in turn to explain how they would use the system to perform it and what are the challenges they face.
  • Once you get inputs from the primary speaker,you can open the discussion for other speakers to provide inputs on what they feel can be improved or the challenges they face.
  • I know this goes back to the situation of them telling you what they want but it might also give inputs on how many people feel the same way on a certain user flow and what functionality is critical and what is the expected behavior.
  • You could also ask for anecdotes about how their users had specific complaints about the functionality and the workarounds they had to employ.

That said there are some concerns from the above method

  • Users might not be willing to talk freely or might look for other users for approval which might skew your results.
  • Users might not be able to remember the steps they take to handle the different user flows.

I would also recommend looking at this study on how Kepler used group usability testing to understand issues ( rather different from your needs but it might give some inputs)

The test included the following usability activities (in temporal order):

  1. User Profile Survey – 20-30 minutes

-Technology Expertise

-Software Inventory

-Job, Education, and General Demographics

  1. Basic Tasks Exercise – 1 hour

-Run an existing workflow, then add one output/display component. -Create a new workflow (simple graph plot of data).

  1. Usability Issues Discussion – 1 hour

These activities were embedded the training program. The user profiling activity was done at the beginning of the training workshop. As part of the training, participants were given an introduction to key terms and concepts, and also performed several familiarization exercises with the Kepler software. After this introduction to Kepler, the group usability testing was conducted.

Participants were given two basic task exercises to perform on their own. These tasks were given as a set of written instructions. Participants were told that observers would be watching and recording issues, and that they might ask quick questions. Users performed the given tasks individually, but simultaneously. The participants performed the following tasks: 1) modify an existing workflow, and 2) create a simple workflow.

Users were seated in one large training room in groups of five, with each group in a somewhat circular pod configuration. They couldn’t easily see their neighbor’s screen without purposefully leaning over to look. During testing, multiple observers walked around and interacted with participants, answering questions, and minimally probing users. The tester/observers included a usability professional, a trainer, and a software developer

share|improve this answer
    
This answer seems to assume that the OP is going to have an hour with the users - but the question stated that there will be no users there, only their managers. They don't use the system, and don't know what challenges there are, they only know "I want my underlings to be 15% more productive because they now have data on their fingertips". –  Rumi P. Feb 13 at 11:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.