2

-This is why I'm asking. Skip to TLDR if you just want what I'm asking- We're in the early stages of getting ready for redesign of a massive messy intranet using modern design principles.

One of the things we'll be doing is asking a group of users to find stuff on the current intranet. The problem that occurred to me is that, in our test situation, the users would know that the content they're looking for is somewhere on the intranet and would thus be likely to keep trying until they found it. In real life, they won't necessarily know if the intranet has that information and may give up too fast or not even think to look for it there in the first place (this happens to me a lot.)

So I was thinking we might randomly ask the users to search for content that isn't there, and we would tell them up front that some of the stuff they'll be looking for isn't on the site. That way in our test scenario, users will be more likely to give up when they naturally would.

TLDR: My boss likes the idea of giving users searches for stuff that isn't there but since its just an idea that I had, she wants me to see if there is anything out there recommending for or against this approach. Do you know of anything? Does this seem like a good idea to you? I googled and couldn't find anything. I don't know the terms.

Improve this question
3
  • Out of curiosity, why do you want to know when users give up on a search? From a UX perspective I would want users to give up ASAP on a search for things that aren't there, and persevere until they find what they are looking for (while minimizing the search effort) on a search for things that are there.
    – uxzapper
    Sep 15, 2013 at 1:02
  • I've been on the user side in this company. It just occurs to me that there's plenty of questions I and my coworkers have had in the past that we didn't even think to try to look up on the Intranet. But if I was sitting in a user testing focus group thing and was asked to look those things up, I'd find them because I'd be directed to look for them. So I'm trying to find a way to compensate for that.
    – methodOverload
    Sep 16, 2013 at 12:44
  • That's exactly the kind of insight that contextual inquiry helps uncover.
    – uxzapper
    Sep 25, 2013 at 2:32

1 Answer 1

Reset to default
3

If you are looking to introduce a bias towards giving up, that's one way to do it. If you want to learn when people give up, give them tasks they can succeed at, but are difficult. Some people will have a harder time with the tasks than others, some will be very persistent, others will give up easily. It's highly individual. Success rate is an important metric. I wouldn't give users a task they could not succeed at.

If you tell them flat-out that the item doesn't exist, they may not want to "play along" and keep searching for it. It will be pointless and they will feel like they are wasting their time.

I might recommend giving them one completely self-directed task at the beginning of the test, where they could search for whatever they want to search for. Some things will exist, some won't (and you will know because you can check their request against your inventory of the site). This will be a better measure of their real-world behavior than tasks they know they can't complete.

Improve this answer
5
  • Your answer suggests I might not have been completely clear. What I mean is, I might, for example, have the users search for ten things on the site and maybe one or two of those things we know aren't there. I wouldn't tell the user at the beginning of those one of two tasks that they're impossible. I'd tell the users at the beginning of the entire sequence that one or two are going to be impossible. You raise a good point about creating a bias to give up. I was also considering rewarding users for finding stuff that is there to mitigate that. I also like your idea about an undirected search
    – methodOverload
    Sep 12, 2013 at 15:11
  • And I like your idea about putting some difficult searches in there. And thank you for the site link. I'm trying to be a developer, a designer, and a project manager when I've never been any of these things. This helps.
    – methodOverload
    Sep 12, 2013 at 15:22
  • @methodOverload, now I understand the username ;-) That's a lot of things to be! Thank you for the comments. Test design is an art and a science. I like to design in a "gimme" task as the very first one just to get the participant comfortable. People tend to feel judged when they are in the hot-seat, so anything we can do to make them feel more comfortable is good. The tasks get more difficult from there. A self-directed task is a nice one to start with, too, because it provides insight into their mental model of how the site should work.
    – LindaCamillo
    Sep 12, 2013 at 17:12
  • 1
    It just occurred to me that your task could be dual purpose. It could help us understand what the user would expect to find on our intranet not just how they could expect to find it.
    – methodOverload
    Sep 12, 2013 at 17:19
  • Indeed - you could also request analytics of the top unsuccessful searches. That can quite revealing as well. I learned that people could not spell "motorcycle" when searching for motorcycle insurance; and that they thought of time off not as "PTO" but as "vacation". ;-)
    – LindaCamillo
    Sep 12, 2013 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.