I'm trying to come up with a strategy for putting together focus groups while designing a web application, but I'm starting to wonder how often is too often to do focus groups? In my mind, they would be beneficial after just about every step of the design process.

Here is my process so far:

  • Step 1.) Identify client's goals, abilities & project scope
  • Step 2.) Focus groups with target market to discover useful features and ideal layout
  • Step 3.) Compose wireframes
  • Step 4.) Create static mockups (potentially with filler content and minor animations)
  • Step 5.) Focus group for usability and visual style
  • Step 6.) Seemingly endless development process...
  • Step 7.) Focus groups with Alpha version
  • Step 8.) Further development, bug fixes, probably more usability features
  • Step 9.) Beta Release
  • Step 10.) In-app user feedback if possible and/or more focus groups
  • Step 11.) Step 8 all over again
  • Step 11.) Launch

So what do you think of this process? Is any particular step unnecessary or missing? Too many focus groups?

3 Answers 3


Wow - this looks really nice. You've done your homework well, I can tell you that. However there is only one step I think you need to think through more. And it's this one:

Step 6.) Seemingly endless development process...

Depending on resources and scope, this step can be all from one week to 1½ years. And letting your focus group wait 1½ years doesn't do the trick. Try to break it down in 3 week sprints and let your focus group see where you are, and correct along the way.

If not you might end up with a project starting in 2008 for a Swedish bank, who didn't care about mobile banking, and five years later was scrubbed. Over 100 million USD out the window. Use Google Chrome to translate from Swedish.

Otherwise you've done a super job!

  • Thanks for that. :) I actually think I can empathize with that Swedish bank from one of my projects. You say "break it down in 3 weeks sprints", but what do you do in portions of the project during which changes and improvements aren't really visible to the user? Would it, instead, be better to set out milestones throughout a project so that there are always specific features/modules for focus groups to test? Feb 22, 2013 at 15:06
  • @dom Milestones are fine if they come regularly at three weeks, but don't use the waterfall project model since it's bound to fail. Use agile methodology instead Feb 22, 2013 at 18:19

I'd say that it all depends on what you mean by focus group.

If you're going to sit down with your users and ask them what they think, and what they want, I would not give this too much of your energy. If users tell you it's wrong they are almost always right, but if users tell you what's wrong and what should change, they are almost always wrong (to paraphrase Neil Gaiman). You can use these to generate ideas at the start of a cycle, and as a sanity test at the end, but don't tie yourself down to the result in the project planning.

If you're going to observe your users and do formative evaluation, I'd plan as many sessions as you can get away with (but not with groups, with individuals). Every week during the wireframing stage is not a luxury. Basically, if dropping one focus group session with five people gets you five individual user tests, I'd drop all of them.

In your plan, step 5

Focus group for usability and visual style

sounds the most worrying to me. Usability and design should not be evaluated by focus group. You can get good feedback from users, but not usually by asking them what they would change, and not by putting them in a group. There's a real danger of devolving into design by committee here.

  • Hmm... your response interests me. I've ran failing projects which I think failed due to insufficient user study. It's obvious to me that that's important, but you seem to suggest that it's counter-productive. I understand how you can overwhelm a project and head off in the wrong direction by doing everything the users say, but if receive user input correctly and take it as data rather than direction, how can you go wrong? Thanks. Feb 22, 2013 at 15:02
  • Quite the contrary! User study is the absolute life blood of UX design, as far as I'm concerned. The more tests the better, at any stage of the project. I just don't have much faith in focus groups. You should separate your users, ask them whether it's wrong but not why, and observe them interacting with your application/wireframes (As soon as you've filled one page of printer paper, you can start testing).
    – Peter
    Feb 22, 2013 at 15:04
  • It just occurred to me that there is actually a technique of giving a small number of users a complex task to solve together using your application. But the goal there is not get them to voice their opinion but to make it easier to observe their thought processes, since they will talk out loud more naturally. But you're still observing your users, not asking them direct questions.
    – Peter
    Feb 22, 2013 at 15:07
  • Yeah, this is a method I'm familiar with and really like. If I'm building a chat client, telling the user to "Send a chat message in bold text" would probably produce useful results. Do you know of any links to articles describing your ideal user study? Feb 22, 2013 at 15:11
  • 1
    My eyes were first opened to the subject by reading 'Don't make me think' by Steve Krug. I've recently started teaching from the slides of the MIT usability course which are really good. 'Paper prototyping' by Carolyn Snyder looks like a good book. And here is a nice article on A List Apart.
    – Peter
    Feb 22, 2013 at 15:19

It depends what you mean by "focus group". If you mean just asking a group for general feedback on designs our concepts, then no, I don't think that's terribly useful to do repeatedly.

Focus groups are good for high level research at the start of a project when you're investigating what problems exist and how and why users are generally performing their tasks. They can also be used to validate personas if one on one interviews weren't possible during persona generation.

What they are not so suitable for is optimizing interfaces - the participants are not using the application, so feedback is at best hampered by lack of weight (we don't know which problems our users mentioned actually stop then completing tasks), at worst totally irrelevant (users start discussing things like graphic design).

Use moderated user testing (where users each individually - not as a group - attempt a particular task on a prototype) to optimize your interfaces instead. Visual design can be tested through "five second tests" with twenty our so individuals (possibly remotely), where participants are briefly exposed to a mockup and then asked questions about what they think the organization does and the alerts of values they think the organization possesses. Visual design at interface level is validated through the aforementioned user tests, not by asking users their opinions but actually observing whether visual metaphors, iconography choices and typography support our hinder the successful completion of tasks.

Finally, I would recommend budgeting for analytics / behavioural tracking features and reserving at least one post release iteration for improvements (our, even better, a continual improvement process if the sort Benny suggests above).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.