usability testing and target group

Question

I intend to conduct a usability study on a website (say an elementary school) to find out about ease of use, learnability etc., the first group of prospective users I would approach are parents for taking a usability survey, as they represent the broader population (target users).

Do you believe it is practical to gather data from a different audience, say college students, who may not be using the website at all, but at least have an insight about features and content of a regular Web 2.0 website, and what is expected when a user would access the elementary school's site?

Answer 1

Yes, if the cost / difficulty of gathering data from college students is low and easy but needs to be kept in check if you are still going to test the actual target audience. It also depends on how many rounds of testing you plan on doing and time management. Project creep be lurking unless user testing is done in parallel?

Yes, why not so long as this is not so far from normal procedure with whomever you work [if any] that the potential fallout might later lead to consequences not worth taking such a risk. There is a lot to be gained from testing non-stakeholders. It is actually economically in your interest to test the group that provides the most feedback from the least resources. In fact the target audience could never end up being tested; which while extreme is not necessarily bad. Hallway testing between expensive programmers / QA will still almost always being cheaper / productive than user testing [unless you can deploy code and A/B test easily like Google to millions].

The question is really how similar are test audience XYZ and target audience ABC? And, thus how much valuable feedback will XYZ be able to provide at what cost?

Pros

• Diversity in testing audience(s) will find many more unexpected usability problems in the rare 10% of use case scenarios.
• Two user group tests of 3-5 people is fundamentally better than a user test made up of 20 users, and may even cost less but do not rely on that.
• Multiple rounds of testing is magnitudes better than one round. It allows you to test your testing methods, is a group of related questions or a specific question falling flat and not creating constructive feedback? It also allows production to fix bugs which will render a smoothing user testing experience and get to the nitty gritty but also test whether the first groups feedback was actually constructive / implementable.

Cons: Dun dun. Dun....

• Feedback from tech savvier college students may miss more common issues [90% of use case scenarios] that less experienced may be tripped up by. Hopefully you are good at putting at different hats on and empathize well with technophobes well enough that these are not the problems primarily that user testing is being intended to catch.
• Stakeholder motivation, a group of parents on the other side of the world will probably have better aligned interests than a group of college students. Unless you do not randomly sample and actually test a subset, ie parents at college but not necessarily parents at said elementary school.

Answer 2

I'm fond of applying Steve Krug's maxim "Recruit loosely and grade on a curve".

If you can't find your ideal users feel free to recruit other folk, but apply more weight to the feedback of people who are closer to your target group.

Answer 3

Both the above answers are great. I will add that when possible I add extreme edge cases to my test panels. A trick I learned by accident.

I was testing a mobile banking app for business people. I usually order panels from an agency that recruits them for us. The first bunch was mistakenly switched with a panel of non-business people for another project. Anyway - when they arrived we decided to just go with the test anyway. We got so many great questions and insights from the "wrong group" that we improved the test for the right one. In this case it was especially important because our business people were very busy and hard to recruit - so it actually paid off optimizing the test itself this way.

It can be hard convincing stakeholders to fund or make time for non-core profiles but it is worth it.