Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I'm trying to come up with a strategy for putting together focus groups while designing a web application, but I'm starting to wonder how often is too often to do focus groups? In my mind, they would be beneficial after just about every step of the design process.

Here is my process so far:

  • Step 1.) Identify client's goals, abilities & project scope
  • Step 2.) Focus groups with target market to discover useful features and ideal layout
  • Step 3.) Compose wireframes
  • Step 4.) Create static mockups (potentially with filler content and minor animations)
  • Step 5.) Focus group for usability and visual style
  • Step 6.) Seemingly endless development process...
  • Step 7.) Focus groups with Alpha version
  • Step 8.) Further development, bug fixes, probably more usability features
  • Step 9.) Beta Release
  • Step 10.) In-app user feedback if possible and/or more focus groups
  • Step 11.) Step 8 all over again
  • Step 11.) Launch

So what do you think of this process? Is any particular step unnecessary or missing? Too many focus groups?

share|improve this question
add comment

3 Answers

Wow - this looks really nice. You've done your homework well, I can tell you that. However there is only one step I think you need to think through more. And it's this one:

Step 6.) Seemingly endless development process...

Depending on resources and scope, this step can be all from one week to 1½ years. And letting your focus group wait 1½ years doesn't do the trick. Try to break it down in 3 week sprints and let your focus group see where you are, and correct along the way.

If not you might end up with a project starting in 2008 for a Swedish bank, who didn't care about mobile banking, and five years later was scrubbed. Over 100 million USD out the window. Use Google Chrome to translate from Swedish.

Otherwise you've done a super job!

share|improve this answer
    
Thanks for that. :) I actually think I can empathize with that Swedish bank from one of my projects. You say "break it down in 3 weeks sprints", but what do you do in portions of the project during which changes and improvements aren't really visible to the user? Would it, instead, be better to set out milestones throughout a project so that there are always specific features/modules for focus groups to test? –  dom Feb 22 '13 at 15:06
    
@dom Milestones are fine if they come regularly at three weeks, but don't use the waterfall project model since it's bound to fail. Use agile methodology instead –  Benny Skogberg Feb 22 '13 at 18:19
add comment

I'd say that it all depends on what you mean by focus group.

If you're going to sit down with your users and ask them what they think, and what they want, I would not give this too much of your energy. If users tell you it's wrong they are almost always right, but if users tell you what's wrong and what should change, they are almost always wrong (to paraphrase Neil Gaiman). You can use these to generate ideas at the start of a cycle, and as a sanity test at the end, but don't tie yourself down to the result in the project planning.

If you're going to observe your users and do formative evaluation, I'd plan as many sessions as you can get away with (but not with groups, with individuals). Every week during the wireframing stage is not a luxury. Basically, if dropping one focus group session with five people gets you five individual user tests, I'd drop all of them.

In your plan, step 5

Focus group for usability and visual style

sounds the most worrying to me. Usability and design should not be evaluated by focus group. You can get good feedback from users, but not usually by asking them what they would change, and not by putting them in a group. There's a real danger of devolving into design by committee here.

share|improve this answer
    
Hmm... your response interests me. I've ran failing projects which I think failed due to insufficient user study. It's obvious to me that that's important, but you seem to suggest that it's counter-productive. I understand how you can overwhelm a project and head off in the wrong direction by doing everything the users say, but if receive user input correctly and take it as data rather than direction, how can you go wrong? Thanks. –  dom Feb 22 '13 at 15:02
    
Quite the contrary! User study is the absolute life blood of UX design, as far as I'm concerned. The more tests the better, at any stage of the project. I just don't have much faith in focus groups. You should separate your users, ask them whether it's wrong but not why, and observe them interacting with your application/wireframes (As soon as you've filled one page of printer paper, you can start testing). –  Peter Feb 22 '13 at 15:04
    
It just occurred to me that there is actually a technique of giving a small number of users a complex task to solve together using your application. But the goal there is not get them to voice their opinion but to make it easier to observe their thought processes, since they will talk out loud more naturally. But you're still observing your users, not asking them direct questions. –  Peter Feb 22 '13 at 15:07
    
Yeah, this is a method I'm familiar with and really like. If I'm building a chat client, telling the user to "Send a chat message in bold text" would probably produce useful results. Do you know of any links to articles describing your ideal user study? –  dom Feb 22 '13 at 15:11
1  
My eyes were first opened to the subject by reading 'Don't make me think' by Steve Krug. I've recently started teaching from the slides of the MIT usability course which are really good. 'Paper prototyping' by Carolyn Snyder looks like a good book. And here is a nice article on A List Apart. –  Peter Feb 22 '13 at 15:19
add comment

It depends what you mean by "focus group". If you mean just asking a group for general feedback on designs our concepts, then no, I don't think that's terribly useful to do repeatedly.

Focus groups are good for high level research at the start of a project when you're investigating what problems exist and how and why users are generally performing their tasks. They can also be used to validate personas if one on one interviews weren't possible during persona generation.

What they are not so suitable for is optimizing interfaces - the participants are not using the application, so feedback is at best hampered by lack of weight (we don't know which problems our users mentioned actually stop then completing tasks), at worst totally irrelevant (users start discussing things like graphic design).

Use moderated user testing (where users each individually - not as a group - attempt a particular task on a prototype) to optimize your interfaces instead. Visual design can be tested through "five second tests" with twenty our so individuals (possibly remotely), where participants are briefly exposed to a mockup and then asked questions about what they think the organization does and the alerts of values they think the organization possesses. Visual design at interface level is validated through the aforementioned user tests, not by asking users their opinions but actually observing whether visual metaphors, iconography choices and typography support our hinder the successful completion of tasks.

Finally, I would recommend budgeting for analytics / behavioural tracking features and reserving at least one post release iteration for improvements (our, even better, a continual improvement process if the sort Benny suggests above).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.