User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
I'm planning for Live User Testing of my tool but I'm wondering what the general concensus is regarding inviting the users to come back for a followup session.
This may be good because they will already be familiar with some aspects of the tool and provide feedback on any implemented improvements.
Or it may be bad because they will have the bias of familiarity.
Is it better to invite a new group in every time?
If your testing methodology is to drop the users in front of the tool cold with no instructions, in order to test how intuitive learning the tool is, then you need new testers. Otherwise, it's likely a matter of convenience: are they available to come back and test again?
If this is an upgrade to an existing tool that has a user base, it doesn't matter at all. Bring in experienced users and new people.
It also depends on how complex your introduction is. If they need 4 hours of training just to turn the prototype on, all of your testers are already biased before they even see the product. Reuse them and save the training time. However, even if you don't plan to reuse them, you might bring one or two back to ask them questions like "You said X was bad, I changed it to Y, does that help you?"
