If participants are forgetting what you asked them to do during a usability test, that could indicate issues with the design that is being tested. However, it could also indicate ways in which you can improve your usability research methods.
For example:
- Are these appropriate participants?
It's ideal to ensure diversity, particularly if your product truly has a diverse audience. However, the majority of your participants should reflect your target users. If you're trying to test a tool for booking flights from A to C via B and nobody in your study has ever used the internet to book a similar flight, it'll be harder for them to imagine that they're in that situation and try to pretend that they're doing what they would "normally" do. You also won't get feedback about how your tool compares to other tools that they've used for booking flights which can be a very useful complement to your competitive research.
- Are you using the right compensation?
If your ideal test subject is a busy, high income professional and you offer a $5 gift card to Starbucks as a reward for a half hour study, it's likely that you'll have a tough time getting the "right" participants. If your participants seem like they aren't putting much effort into attempting the tasks, giving detailed feedback, etc. that can be a sign that they aren't very motivated or aren't very interested in the topic. People who travel regularly will be often be more excited about testing out a new travel website and giving feedback than people who don't.
There are also a lot of "career" participants who will sign up for a study about anything and claim to fit your demographic when in actuality they don't.
- Are you using a mixture of "closed" tasks and open-ended tasks?
A task like "Book a flight from A to C via B" can be useful for testing a specific part of an interface, but if every study task is like that, you're missing out on valuable insight that you only get by watching what people would do without that structure. For example, "Do you have two friends or family members who live in different cities? Okay, what cities do they live in? Great. Imagine that you want to go visit your friend from college but pause for a layover near your mom so that the two of you can have lunch. Show me how you would do that from this screen."
That task will be easier for participants to remember because it has context - people they know. It is testing your desired specific scenario. It can also provide new, useful insight... "Oh, I didn't realize they'd try typing PDX instead of starting to type POR for Portland."
Open-ended "tasks" / interview questions are great for learning about assumptions. For example: "Tell me how you typically travel." "Tell me everything you notice about this page." "What are all the things you could do on this website?" "When might you use a website like this?" (at home, at work, on the bus, etc.) "What other websites like this have you used?"
- Ensure that you know why they aren't completing the requested task.
If you ask a participant to try something specific and they seem to be aimlessly wandering around, that can indicate that they forgot. If so, a gentle "What are you trying to do right now?" can help steer them back towards focusing on what you want them to attempt.
However, aimless wandering can also indicate that a participant does remember but is completely lost in the navigation. If they do remember the desired task, try questions like "Where would you expect to find the place to look for flights?" "If you really needed to fly out tomorrow, what would you do at this point?" "Is there anything you see here that seems like it's confusing or not what you'd expect?"
Not understanding terminology can be one motivation for seemingly aimless wandering. For example, if your participants are not familiar with travel, they may see a form that requests that they enter their desired airport code. However, if they don't know what an "airport code" means, they probably aren't going to try using that to search.
- Most people are not primarily auditory learners.
I'm an auditory learner, so if you tell me something out loud, I'll often be able to remember it. However, the majority of humans are visual learners or a combination of visual and audio learners. Some humans learn primarily by actively doing something.
If you absolutely can't give participants an open-ended task or a task that directly relates to their life, they'll have a much easier time remembering information if it's written. Print a packet with the tasks, write them down (or have the participant write them down), provide a presentation or file that lists them, etc.
- Are the tasks themselves confusing?
Participants can get confused by the way you word a task. ("Does 'book a flight home' mean the place I'm living this month, or does it mean my permanent residence?" "What does booking a flight 'via Chicago' mean?" etc.
Participants can get confused by the tasks themselves. ("I don't HAVE friends in multiple cities. Why would I ever care if a plane passed through city B or city C?")
Participants can get confused by what defines 'succeeding' at the task. ("I just got to a screen that shows me all the fights that go from Seattle to San Francisco and stop in Chicago. Hooray, I'm finished!")
Before you have anyone come in for a study, make sure you run the study tasks by multiple people to make sure that they understand what you are asking, how they'd know if they were finished with the task, etc.
...
For more on all of this, I'd recommend Interviewing Users.
Also, unless you're making a product/service that's only intended for men, you'll make this researcher terribly happy if your generic user is not always referred to as "he". ;)
my following generation would just breath through for a few seconds
- Don't generalize.