How should I user test teenagers? (Under 18)

Question

I'm currently working as a UX design intern at an organization that revolves around motivating teenagers. One of my big projects for the summer is to help my supervisor get our user testing program off the ground.

Our problem is that most services that provide testing only target down to age 18, whereas our age group is more in the 13-17 category. We've contemplated calling in users for recording, but we're worried that won't be a great representative sample, as we have a national membership base.

Are there any solutions that I've overlooked? Is my fear of non-diversity baseless? We've never done user testing to my knowledge, so this is a very new process for us.

Answer 1

Well, not to shut you down or avoid answering your question, but there are usually a bunch of legal consent problems around testing with subjects under 18 yrs old. You would have to have the parents give consent since the kids can't. You might have already overcome this hurdle since your user base is in the 13-17 category, but I just remember my professors stressing us to only use 18+ yr olds during my university usability testing.

Now, to the testing!

I'm not sure exactly what you mean by user testing, but I'm going to take it as what a user thinks about when they interact with your product. It sounds like you are dead set on face-to-face testing, but surveys are a great way to get started. You've just got to figure out the right questions.

The best and most efficient way for a national group to get usability testing started is going to be online things. You will get the most diverse and representative sample. I mean, if it's a website you could theoretically test your entire population (user base)! This is also the cheapest way to test since it involves you doing what (I assume) you are already paid to do: sit at your desk. There are a ton of great online survey tools from surveymonkey up to really complicated things like surveygizmo (the most complicated that I've used). Then, even above that there are services that will implement and collect responses for you. Online surveys have a low start up cost and a relatively high pay out if you do it right.

Now, if you really really can't do it online then you are going to have to take some hits to your diversity and call them in. This is expensive and time consuming because you can only test one person at a time. You're also going to be limited to the people who have the ability to come to you/your facility. And, since they are under 18, mom or dad will probably have to bring them to you. But if there is some class you teach or some group that you interact with on a regular basis because the program offers face-to-face interaction then that would be the best time to perform some usability test. Now the big problem with this type of testing is figuring out how to standardize data across subjects. You need to figure out a base set of test data to collect before the tests even begin. Then you have to make sure you test each user the exact same way in order to run statistics on all the data. You cannot lump data together if it doesn't contain the same variables. This brings me back to surveys. You don't get the fine-grin data that you would potentially get from observing users, but its a much much easier and productive way to get into user testing.

As far as diversity goes, try to figure out what the diversity of your user base is currently and then see if it's feasible to get a sample like that in your area. You are still missing the rest of the country so your fears are founded I'm afraid. Even with a skewed demographic you can still come to some useful high-level conclusions. Just don't try to delve too deep into the data because it might take you in the wrong direction!

Answer 2

Suggestion @LukePatton: Get in touch with a kid and ask him to contact some of his fellow students.
I suggest you contact my 16 years old son agustin dot lanus at gmail dot com (I expect this not to be illegal in StackExchange).
As kids are usually computer-savvy, you can ask them to do things that usually would require some technical skills like install a screen recorder.
Offer them a bunch of dollars via PayPal. Ask the coordinating kid to collect parental authorizations and send them to you via email, scanned. Anyway, laws in this country are much safer for you.
I recommend these children because in their school some of them study IT on Communications, Communitations Media, and other specializations that might be interesting for diversity.

Don't filter too much. As @maprules1000 said, a skewed demographics is also useful for finding issues of your UI. Especially, if you are aware of it.
Be wary of publishing your site before having it tested. There is nothing you can't fix later except deceived users that won't return, never.
In addition to recording the interactions and their voices, I strongly suggest that you schedule with each one a post-test interview, with open-ended questions, keeping in mind that it's an "alien2 audience from the standpoint of grownups.

You can also set your application to record and transmit the user's interactions, like where they clicked or typed and when.
Test with a small set of kids, recycle the results, and then do another test.
If you have no previous experience then follow the really simple-to-understand guidelines (or should I say detailed instructions?) in Chapter 9 of Steve Krug's "Don't make me think" book.

After, you can set A-B tests and use the whole audience as test subjects, for free.