4

I will be doing the second testing soon. The question is: Is it possible to invite to the second testing those people who passed the first testing?

2
  • Why just the people who have passed the first test session? Why not the people who didn't pass it the first time? What is the main objective of running multiple testing sessions with the same users? – Michael Lai Feb 19 at 4:07
  • 1
    Maybe my English isn't that good, but "passed" suggests the users have been tested, right? If that's the case, in please check your attitude. The users are testing the Software. The Software can pass or fail - never the users . If a user can not fulfill a task with the Software it is the Softwares fault! It is NEVER the users fault. – htho Feb 24 at 4:44
4

No.

People learn.

If the software is similar to the first time, your users will have learnt how it works.

2
  • 2
    But you can ask them if the change was an improvement. – allo Feb 18 at 23:18
  • That's a different kind of research. Usability tests aren't assessing whether people like something. They are assessing whether it can be used (easily). – PhillipW Feb 19 at 9:14
3

Yes, but their input should be treated differently than users who were not a part of the first round, and you certainly still require a group of users who have not previously tested it.

To expand as requested in the comments, results from users of a previous test should be strictly separate from new users, and will be handled more anecdotally. i.e. a user in the second test may report back that "the options selection feature works, and I was able to use it the first time, but on the second use, noticed it's very tedious when you wish to turn off or on all options. The process could be simplified by adding a "toggle all" switch at the top of the group". The feedback could be anything- the point being a second time tester may find additional details a first time tester never will. It should change or affect your goal %'s for a first time usability test, but more feedback rarely hurts.

1
  • Can you elaborate on this? How do you suggest processing the results from users that also have done the previous test? – jazZRo Feb 19 at 11:30
3

In learning we have received from PwC Experience Center on Qualitative Usability testing, it was noted participants should be new ones.

Advantages of this are:

  • if they are first time here, they are acting as real users, and not design test professionals
  • they don't have any experience with the brand or app (positive or negative), therefore they are more honest and precise with giving answers after
  • it's statistically not totally correct to mix findings from reused and new participants

Therefore, I would say it's possible (as it is practical sometimes). But data taken from tests should be taken with a grain of salt.

2

It depends on which market you're trying to test.

We can distinguish the following market segments, and probably many more:

  1. users who have never experienced a product in this category
  2. users of a competitive product who have never experienced your product
  3. users of a competitive product who have experienced your product
  4. active users of your product
  5. disengaged users of your product for 1+ weeks
  6. disengaged users of your product for 1+ months, etc.

All of these groups might be important to test, depending on what your goals are. For example, if you're trying to figure out how to re-engage inactive users, you will probably focus your testing to optimize group 5, but also test other groups to validate that you haven't deteriorated their experience.

Back to your question: can I invite the same participants back for a second round of testing?

Of course you can. The problem is that your first test might have moved that user into a different market segment. For example, if your goal is to test users from market segment 1 — users who have never experienced a product in this category, then the participants from the first round of testing have now experienced a product in this category and are somewhat familiar with it, so they now belong to segment 2 or 3, depending on how much you've revised your UX.

0

In my opinion this is fine as long as the 1st test doesn't have any learnings that could be used in the 2nd test (unless you want that). There can be also some benefits as they are more familiar and comfortable with you they are more likely to share more. If you are conducting user testing regularly it's likely to happen as some participants like the fact their feedback is being used to make improvements.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.