38

I did a lot of usability tests and sometimes users came to me saying:

Wow. When you said TEST, I thought I should study more.

Do you notice the MORE? It means that they studied something! I always explain that the test is about the system, not about the user, but it doesn't solve the problem.

Do you have this same problem? How can I ease the minds of my testers? Should I explain more, use another name for the test, or something else?

  • 28
    It's kind of ironic. We call it user testing, but we're not testing the user, we're testing the product. We even tell the users "we're not testing you, we're testing the product, nothing you do is wrong, just say what comes to mind." Why isn't it called product testing? – Majo0od Jul 27 '15 at 13:25
  • Ironic indeed. :) Do you use this terminology? – Giu Vicente Jul 27 '15 at 13:27
  • 7
    I think "usability testing" is already very clear we're testing the usability of the software, not usability of users... users aren't being used, after all. Minor grammatical differences you can put in brackets to clear up next time you send out the invites. ;) – CleverNode Jul 27 '15 at 14:31
  • 1
    @Majo0od I guess the term "user testing" is meant to emphasize that the users are testing the product, not developers or Q&A. – Sebastian Negraszus Jul 27 '15 at 15:48
  • 2
    Why not call it a test-drive? People know that in a test-drive it's the product you're trying out that's supposed to perform, not you. – Matti Virkkunen Jul 27 '15 at 19:14
44

This is something worth paying attention to. Aside from the highschool/college exam flashback anxiety you talk about, the Hawthorne Effect suggests that "individuals modify or improve an aspect of their behavior in response to their awareness of being observed", and some might argue that it's practically impossible to "observe" a natural use of a product or website (without resorting to secret recording). That said, there's still at least some value in testing users, so we should probably strive to mitigate this effect with language.

In the approach, you could ask people to take part in a "Website Evaluation", rather than a "Usability Test". You could play around with other words like "assessment" and "analysis", always making sure that the website or app is the subject of the sentence.

When it comes time to carry out the "website evaluation", read them Steve Krug's Usabilty Script, part of which reads:

We're testing a web site that we're working on to see what it's like for actual people to use it.

I want to make it clear right away that we're testing the site, not you. You can't do anything wrong here. In fact, this is probably the one place today where you don't have to worry about making mistakes.

We want to hear exactly what you think, so please don't worry that you're going to hurt our feelings. We want to improve it, so we need to know honestly what you think.

  • 7
    I second the notion "Website evaluation." That makes people feel as though the site is being evaluated instead of themselves. – Majo0od Jul 27 '15 at 14:49
  • 1
    Note that there is debate over whether or not the Hawthorne effect actually exists. – nekomatic Jul 28 '15 at 14:39
  • 1
    I participated a couple of usability test where I was told that a certain thing was being tested (font choices, color schemes), when actually the experiment was designed to control for those things and actually test something else (a functional choice). I only found out when I went back to see the results of the study later. I don't know if that's really a good idea or not, but clearly an attempt to counter this effect. – mattdm Jul 28 '15 at 16:22
  • The actual word 'usability' is industry jargon and won't actually mean anything to a chunk of the population. – PhillipW May 8 '17 at 19:59
8

It's all in the phrasing and perception of who is being targeted. Below are a few suggestions:

Please help us evaluate our new online ordering system

Try out our new online ordering system, let us know what you think, and get 5% off your next order!

  • +1. I even stay away from 'evaluate'. "I was wondering if you'd be interested in trying out a new system and seeing what you think." Of course the usability test itself is more structured than that... – Alex Feinman Jul 27 '15 at 14:48
  • @AlexFeinman Thanks! I've updated my answer to include another example – MonkeyZeus Jul 27 '15 at 14:53
7

Depending on the users involved, I've heard usability tests called "Usability feedback sessions" or "usability sessions". (This usually comes up on meeting invites and other 'impersonal' communication channels.)

I've also cut to the chase and called invites things like "Login Form Project -- feedback from Jane Doe". No 'testing', no 'usability'.

In the body of the invite, I'll use a phrase like "I was wondering if you'd be interested in trying out a new system and seeing what you think." Of course the usability test itself is more structured than that...

(During the session, if the user is still hung up on it being a 'test', part of my opening patter is "remember, this isn't a test of you--you're helping us test our software.")

5

Do you notice the MORE? It means that they studied something! I always explain that the test is about the system, not about the user, but it doesn't solve the problem.

"Usability Testing" is, in itself, actually terrible UX when presented as such.

Do you have this same problem? How can I ease the minds of my testers? Should I explain more, use another name for the test, or something else?

Go back to basic UX principles. The very first step: stop thinking about these people as "testers." Your context to their activity is they are helping you [usability] test your software. That is not a familiar context to them, not a context or role they would perceive themselves in or desire to be in normally, nor possibly one that even makes sense to many of them ("usability" is essentially a technical term, even to the point of being jargon, despite being based in a standard english word). A paid software tester is a "tester." Someone else is just trying to do their job/complete their task while being asked to additionally do something for you too, now. From your perspective these may be one and the same and redundant, but that can appear very differently from other viewpoints.

Communication is key here: you need to make sure you are appropriately conveying what will be involved in the activity and what is expected of them, as well as what they can expect and can expect to be doing. I think @dennislees has a completely valid point in referring to Hawthorne, but I think this actually goes even further.

You need to re-home this into an appropriate context for them when you present it to them even if internally you continue to refer to it as it pertains to your own context (usability testing).

What are they here to do? Not in terms of what you want them to do, but in terms of how the activity will play out to them?

Evaluate is a good term. Feedback is also fairly good.

They are here to help you, but it's important to not bog down on what that means to you (which is more where you're currently at). Focus on the things they will be doing and can do that will be helpful. People who are actually eager to help (this essentially goes back to the Hawthorne effect) often try to anticipate needs, which can be problematic both in terms of being a stressor on them as well as not always fitting well with your actual goals.

Also focus on the state of mind you want them to be in when doing this. Play with may even be an appropriate phrasing (perhaps backed with other language to still indicate the need for feedback and what the feedback should pertain to) in order to alleviate any sense of stress.

Are you going to have them do a standard activity with your (new/update) software, as opposed to their normal software (or your current version)? In that case, focus on the activity.

For example:

We'd like to present you with a special opportunity to run a print job on our latest software—before it's widely released! All we're asking for from you is if you could please take the time to give us some feedback on how you feel about our new file uploader. This is simply answering three quick questions and should add on no more than two minutes to your normal print job processing time. If you have some time to answer some other quick questions—or if you have any other thoughts on what you'd like to see improved, or anything you found frustrating—that would be great too!

(note: this is just a quick jot to serve as a showcase)

This sets some clear expectations: they're just going to be doing something they've already done and are familiar with, it's just going to be in a new version of the software. It also sets a single specific feedback goal as an "exchange" task (evaluating the "file uploader" specifically, not the software overall: this could be set differently on different sample populations). This helps initially limit scope so there is a basic focus to work from, and so that the expectation of the task size is clear. It also implicitly cues that the changes are mostly related to the "file uploader", in this case (even if there are other changes), which helps reduce fear that the overall changes to the software will be too overwhelming and make it too unfamiliar to comfortably perform the usual task in the usually expected manner/time frame.

It even includes some time information so that no one is concerned that they're being asked to devote an indefinite amount of time. Indefinites are one of the worst things to build into a task expectation, in terms of lowering desire to commit to it. The more you can solidify expectations of the task activity and surrounding parameters/needs, the more comfortable it is to commit to performing it.

Note that I don't say "test" anywhere in that example. They are not "testing:" they are running a real print job. If this is not the case, you will need to clearly call that out, such as referring to it as a "mock" print job, but I generally would recommend trying to fold this into a working process model.

You can also toss in verbiage on how all of this is to help them (set another reason for why they might want to take the extra time this involves over their normal procedure), but you don't want to make it too long in the process.


Steve Krug's usability script as linked by @dennislees is a great example, but I think it's important to understand why it is presented in the way it is. This all comes back to perspective and context, as well as appropriate language/communication choices. It can be fine to use industry standard terminology, so long as it is industry standard terminology to the people you are communicating with.

You may want to consider usability testing your usability testing communications. While I say this somewhat teasingly for the way it may sound seemingly redundant, it's not a bad thing to at least run such verbiage through someone who does professional copy editing on material that's published to the general public or, more specifically, your intended audience.

1

Don't say 'test'. Say 'interview'. It's easier for people to understand it will be a 'interview' about something.

People tend to believe a 'test' is something they should worry about, when our intention is to have their honest opinion and behaviour.

0

place the "test" as a secondary element to a primary task/goal so the hawthorne effects is projected onto the primary activity and not the test.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.