I was wondering: if participant thinks he has found the solution to a task, do you end the task even if the solution is the wrong one? Eg. you task the participant to find some information, but he doesn't find it, but instead finds some contact info and says 'I would call there and ask'. Will you move on to another task at this point?
My own thought is that this is a good point to move on as participant wouldn't keep looking in real life.
Your goal should be to squeeze out as much information from the participant as you can without leading them.
In this case, I would mark it as a fail but continue prodding him with phrases like "Can you think of any other way you might find the information?". Also, try to figure out why the participant discounted the other possible ways of finding the information in the first place.
Great user research understands both what happened (the user wasn't successful at completing the task) and why it happened (the user didn't complete the task because ... ). Simulating real life doesn't necessarily help you understand why the failure occurred. Understanding why the failure occurred gives you the best opportunity to make a truly impactful change to the design.
If a usability test is qualitative in nature, then I'll probe to determine what makes them think that they were successful. For example, if they're seeing something on the screen that misleads them or falsely indicates that they've completed the task, that is one problem that can be solved. On the other hand, if they're missing a subtle error message that indicates that they didn't complete the task, that's a different problem to solve. If they tell me that they would ask someone for assistance, I ask them who they would contact and what they would say. Depending on the study, I might use this opportunity to give them a nudge in the right direction, and my report would reflect this. ("Four of 7 users were able to complete the task successfully; an additional user completed the task successfully after asking for assistance and being told [whatever I told them].") If they expect to be able to complete a task in a way that you didn't anticipate, then this might tell you that you need to reconsider your design to meet this expectation.
If a usability test is quantitative in nature, then I would mark the task as a failed one and move on. In a quantitative study, it might also be appropriate to note the various ways that users failed. For example, if 20% of users fail the task in one way and 30% fail it in a different way, that's probably important information to convey in your report; or, if 100% of users fail the task but only 20% know that they have failed, that's probably also important information to convey. If there's time after you have gathered all of the quantitative data that you needed to gather, I might go back to the failed task and try to probe to gather qualitative feedback about that failure.
An exception in both qualitative and quantitative research is if failing of this task will block them from being able to successfully complete a future task. For example, if a user has to successfully set up something before being able to complete the rest of the study, I don't want them to fail the rest of the study because they failed at this earlier task. In this case, I'll ask the participant to turn away from the screen or leave the room, and then I will get them to the right point so that they can continue. This is clearly dependent on using a very high-fidelity prototype or working code for your study.
I agree with Jeremy's answer. Though it may not simulate real life, it gives perspective on how close you are to a solution and the importance of other functionality that may assist the user.
For example: If the user can't find the information as you described, they might explain they would use the search bar and find the answer that way. Which would give some indication that the search bar may need to be more prominent.
All information is valuable, even if it's not the immediate response you expected. (Which it never is at least from my experience)
