Are there any benefits to using users who are not in your target market to test your website?
e.g a car website tested by users who don't drive
5
-
I think that'd be like asking some male teenagers to test a website targeted to pregnant mothers that contains guides and tips. They can certainly help with usability and the general look/feel of the site, but they're not much help in terms of testing or using your actual content. It's best to get the target market to test.– MikeJan 28, 2014 at 19:58
-
2@Mike - True, but you also get out of the mold of "I know what I'm doing so I wouldn't try THAT". When we tested a web based translation of one of our old DOS control programs, we found we got more error detection out of the "never used it" group, because they tried things that the more experienced users "just knew they shouldn't".– JohnPJan 28, 2014 at 20:39
-
interesting johnp– user41788Jan 28, 2014 at 21:25
-
1@JohnP "Never used it" is not the same thing as "outside of the target demographic". If you write a new version of an accounting software for example, your old users will be one part of the target group, and accountants who never used it will be another part. If you test it with piano teachers, you'll probably get lots of confusion and not many errors noticed.– Rumi P.Jan 29, 2014 at 9:12
-
@RumiP. We included new users, old users and non users. The new and non users found many more bugs and needed preventions than the experienced users.– JohnPJan 29, 2014 at 14:43
Add a comment
|
3 Answers
You are much more likely get more valuable user research if you research with users that are in your target demographic. Lets take a more extreme example: if I am developing a tablet app that is intended to be used by children with learning disabilities, it would probably not be the wisest move to test this on a motorcycle mechanic. But here our user persona is rather specific.
Okay so how about something else. If I am designing a website for people to buy or sell rare, antique and collectible motorcycles, that motorcycle mechanic now makes a lot more sense. He has a context of background knowledge about what he is going to want to look for when using this website.
Now the average person on the street probably knows very little about antique motorcycles, but they might now a bit about using ebay and other buy/sell type web platforms, so though they might not get me the best feedback, they still might stumble over the price filtering or the sign in/new account and I could learn a lot from seeing that.
So to sum it up, the more specialized your market is, or the more implicit (shared) background knowledge that is needed for that market, the more important it is for you to seek out users that will fit that criteria to test with. Despite all of that though, as long as your demographic isn't extremely specialized, there is still a huge opportunity for you to learn a lot by testing just about anyone. If you can provide enough context for them to know their motivation, you should definitely go for it! And don't forget, test early and often.
Ill leave you with this really fun article about Guerrilla Usability Testing.
Per your example of a car website and users who don't drive, you don't actually drive a website. Therefore, how would the usability of your website be any different if the user does or does not drive?
I think about the only thing that could be an issue by not having the target market test your website would be the actual content itself, as only the target market could effectively tell you whether or not it's up to par.
Functionality, accessability, usability, etc., can and should be tested by anyone. I would think you'd want a very diverse group for that anyway, to catch all the fringe exceptions.
answered Jan 28, 2014 at 19:52
-
1
-
-
@user41788 - Yea ... typically I use 2 groups. One group that is specifically for trying to break my site in any way, shape, form. And one group that tests the content for reliability. Jan 28, 2014 at 21:32
Testing on your target group is of course very valuable. But testing out with your target group is also valuable:
If you test solely on your target group, you will limit (albeit to a good group) the range of scenarios you test against. Non typical users will not know what they're doing and you will get rich data by observing them try and solve a problem, compared to somebody who is experienced in the particular field.
To put it this way, if you were testing how quickly somebody could learn French, would you test on French speakers or non French speakers?
-
What do you mean with "But testing out with is also."? It's very confusing. I read it as "Testing on your target group is of course very valuable. But testing out with [your target group] is also [very valuable]." which doesn't make much sense. Please clarify.– DennisMay 26, 2014 at 18:23
-
Yes, I meant out with [your target group], I have updated the text. I say why in the next paragraph, any answer you only read the first line of will be confusing ;)– S..May 27, 2014 at 16:01
-
I read the whole answer before editing and commenting. It's confusing because you used "But" which is used to introduce something contrasting with what has already been mentioned, except you don't mention anything contrasting. The second sentence is essentially repeating the first sentence, and can be removed. Unless you mean something different with "testing out" compared to "testing on"; in that case I suggest using a better choice of words since the difference isn't clear.– DennisMay 27, 2014 at 16:18
-
In my answer 'but' separates testing with the expected users, or testing NOT with the target users. These are in complete contrast to each other..– S..May 27, 2014 at 16:30