How to design for people not willing to learn?

Question

During quite a few usability tests in the past I have noticed the following: A test person gets a simple task to achieve. Lets say "Book a flight from A to C via B". He should then fulfill the task using the product (in this case a website).

After less than two minutes the test person started to navigate randomly through the entire site: "Where am I?", "This is difficult..", "Really irritating!" etc. After questioning the test person I noticed that he had no idea any more what to do. He literally forgot the task and tried to recover it navigating randomly (as in "go back there to where you thought of that what you just forgot"). More elderly test persons who took their time and ... well yes ... 'accepted' the system did the task with way less problems (using a fraction more time).

Clearly we all want to design easy-to-use intuitive systems but at some stage it is impossible to make it any more clear. Do you know what I mean?

• How do you find the right balance for your product between 'absolutely fool-proof' and 'this takes time'?
• When are users ready to actually learn something and take their time in a non hectic environment?
• Is there any method to put them in to a specific mood so they are more open to anticipate with a product?

Donald Norman wrote in 'Emotional Design' that if a user is happy and feels good (about anything) he is able to cope with problems better because he does not focus only on one possible solution but on more (mental flexibility etc.). But this requires the user to anticipate with the system before using it. So is graphic design the answer to this?

I've been thinking about this for quite a while and would love to read some of your insight. I'd also deeply appreciate some more literature or links if you have some on hand ;-)

Edit: We brief every single test person telling them that we are testing the product and not them, they can take a break when ever they want and take as much time as they need. We have enough time buffer between every user, moderators take turns so they are rested etc. I'd say conditions here are near perfect.

Answer 1

Short answer

You can't design for them.

It can be that your design is bad, or that people really cannot concentrate on the task due to their internal reasons, explained below in the long version. If you have successfully determined that it is the second case, nothing in your design can change how people tick internally. An easier to use application will reduce the unpleasantness, but no amount of usability will turn the experience to be pleasant, and people in this situation will continue to be discontented.

The long explanation of what is happening to your users

When are users ready to actually learn something and take their time in a non hectic environment?

This happens when they care about the task. When they want to do it, they take their time to do it properly. When they don't want to do it, they don't concentrate on it.

The role of the artificial setting

This is an especially hard problem in lab studies (as opposed to ethnographic observations). Your participants are not in your lab because they want to book a flight. They are in your lab because you promised them 10 Euros per hour for a bit of clicking around. Some of them will adopt the goal of doing the task you set them properly, mostly out of altruism (they are in the lab to help you, so they do whatever you need) or pride (they are given a task, and take a pride in doing it right). Others will not adopt your goal; they are there for the consequences of being there (getting the money) similarly to a teenager who is in school for the consequences (not getting a detention) but stares out of the window the whole time.

Causes of not inability to concentrate in natural settings

The above doesn't mean that the behaviour you observe is purely an artefact of your study design. This is just the most probable reason in a lab setting, but it will occur frequently under natural settings too.

The problem is that, in the real world, users don't always want to do what they have to do. This can have different reasons:

• User has an idealistic mental model of the process he is performing, and the current task is not a part of this mental model. In your example, it will be a user who wants to lie on hot white sand under exotic palms. This is part of his dreams, it is what he cares about. Booking a flight to Mallorca is a hurdle on the way to getting to the beach, not part of the goal. Hurdles make him impatient and annoy him, so his brain is distracting him with thoughts like "I will be so glad when this is over and I sit on the plane already". Result: he cannot focus, and doesn't want to.

This is not limited to using IT; I have experienced it in e.g. my young cousin who really wanted to try sewing and was terribly disappointed that she will have spent hours of transferring and cutting patterns and basting fabric before she can sit at the machine for a few minutes of "fun". And also in young developers in small projects who start coding without gathering requirements first, because they want to code, not think about what the user wants.

• User has been saddled with the task by somebody else and doesn't adopt it. Same situation as in the lab. Will most frequently happen with software used for business purposes by badly paid grunt employees where organisation context (low pay, little control over own work) will stifle their motivation to do a good job.

• Cognitive dissonance. Happens when a user has to fill a role or do a task which is contrary to his self-image. Probably not too problematic for you as a flight site if your users are young-ish people from across society. But elder people and people of very high social standing might feel that booking a flight is a secretary's job, and that they are something better than a secretary. They may not be consciously aware of these thoughts (especially the second one), but still they will be subject to cognitive dissonance. Cognitive dissonace is one of the strongest psychological drivers; people have an instinctive urge to avoid its cause just like they have an instinctive urge to avoid biting cold wind. While they try to consciously concentrate on the task, other cognitive processes in their heads are trying to convince them to give up the task.

• Fear. A person has an innate need to feel secure. There are many reasons why somebody can have fear of a task. He may fear failure (self-explanatory). He may fear success (sounds paradox, but has a good explanation which is too long to give here). He may have a fear of being seen to make a mistake, even if the mistake is easily repaired and has no real consequences. He may be afraid of the tools he is using - this happens mostly when he doesn't know them well and has made bad experiences with them in the past. The third reason is frequently found in some elderly users who had their first contact with computers long after they were grown out of the phase of natural curiosity. Some of those people just can't start feeling comfortable with computers, they see them as foreign and unpredictable, therefore menacing. They don't form a mental model of how the computer works, or only a very primitive one, and while using the computer they are always expecting an unpleasant surprise, like being lost in the jungle. While they may want to focus on the task at hand, their brain is in a state of "get me out of here" alarm, and so they can't really concentrate on doing the task. These people will react the same way in the lab and outside of it, probably even worse outside of the lab where there is no other human nearby (the presence of others makes our primitive fear mechanisms relax a bit).

Can you get the users to stop complaining when the application is actually OK?

You should be aware that, should one of these happen, it is not a problem with your application, but a bad fit between the task and the user's personality and/or goals, desires and motivation. Nevertheless, due to a few well known biases, especially attribution bias, and personality factors (some people tend to always have external attribution), they will blame the application. This is a normal process, not arising from any conscious malicious "I need a scapegoat" thought; they really feel irritated by using the application, and say "This is so irritating" without even realising what processes are going on in their own heads. So be prepared to hear a few accusations even of the most perfect application in the world; in the best case, you will learn to recognize unfounded accusations and discount them. In reality, learning this is awfully hard.

Of course, the possibility also exists that your application is indeed confusing and offering a terrible user experience even to people willing to concentrate. I wish I could give you good advice on how to distinguish the two cases, but it is a "soft skill" I cannot express well. Maybe somebody else will feel up to it. But the thing to take away: you can't stop the annoyed users from complaining, you have to find out yourself whether they are annoyed because the application is bad or because they have the wrong reaction to it.

Mood and concentration

Is there any method to put them in to a specific mood so they are more open to anticipate with a product?

There isn't one. The problem is not based on their current mood, but frequently on core "features" of their personality. You probably can't remove these features even with years of psychotherapy. Teachers and employers have struggled with this problem for countless years, and haven't found a solution, except for a few tricks which work in specific cases with susceptible subjects.

This said, a positive mood can certainly increase the willingness and capability of focusing, and in the light cases, this increase may be enough to counteract the decrease caused by the more permanent factors and push the user into a concentrated state. Making the user feel happy will help you, but you have to use generic measures for it (pleasant surroundings, friendly interactions with the personnel, etc.), because you cannot address the source of their unhappiness directly.

In the case of low motivation (second reason) you may be able to change the incentive structure within the experiment, but this still won't help you if the finalized software has to be used in an environment with low incentives (e.g. software for call center agents). If the reason is fear of the "unknown" computer, helping the user to build a correct, simple and non-surprising mental model of your application before use will prevent avoidance behavior, but doing this will render all your learnability metrics useless, beside being terribly time consuming.

answer ends here

I won't go into your first bullet point, it is even more extensive than what I wrote above, and lots of it is just gut feeling.

Answer 2

Hectic, fast and instant. Everything has to be an instant success.

People have a range of personalities and some people work like this.

On one occasion I got someone behaving like this when testing a computer game and I decided to just junk the testing script and use the time to have a chat about why they felt this way about the game they were supposed to be testing and what they would have liked it to have been like instead. It provided some useful insights which went into the usability report.

Answer 3

If participants are forgetting what you asked them to do during a usability test, that could indicate issues with the design that is being tested. However, it could also indicate ways in which you can improve your usability research methods.

For example:

• Are these appropriate participants?

It's ideal to ensure diversity, particularly if your product truly has a diverse audience. However, the majority of your participants should reflect your target users. If you're trying to test a tool for booking flights from A to C via B and nobody in your study has ever used the internet to book a similar flight, it'll be harder for them to imagine that they're in that situation and try to pretend that they're doing what they would "normally" do. You also won't get feedback about how your tool compares to other tools that they've used for booking flights which can be a very useful complement to your competitive research.

• Are you using the right compensation?

If your ideal test subject is a busy, high income professional and you offer a $5 gift card to Starbucks as a reward for a half hour study, it's likely that you'll have a tough time getting the "right" participants. If your participants seem like they aren't putting much effort into attempting the tasks, giving detailed feedback, etc. that can be a sign that they aren't very motivated or aren't very interested in the topic. People who travel regularly will be often be more excited about testing out a new travel website and giving feedback than people who don't. There are also a lot of "career" participants who will sign up for a study about anything and claim to fit your demographic when in actuality they don't.

• Are you using a mixture of "closed" tasks and open-ended tasks?

A task like "Book a flight from A to C via B" can be useful for testing a specific part of an interface, but if every study task is like that, you're missing out on valuable insight that you only get by watching what people would do without that structure. For example, "Do you have two friends or family members who live in different cities? Okay, what cities do they live in? Great. Imagine that you want to go visit your friend from college but pause for a layover near your mom so that the two of you can have lunch. Show me how you would do that from this screen."

That task will be easier for participants to remember because it has context - people they know. It is testing your desired specific scenario. It can also provide new, useful insight... "Oh, I didn't realize they'd try typing PDX instead of starting to type POR for Portland."

Open-ended "tasks" / interview questions are great for learning about assumptions. For example: "Tell me how you typically travel." "Tell me everything you notice about this page." "What are all the things you could do on this website?" "When might you use a website like this?" (at home, at work, on the bus, etc.) "What other websites like this have you used?"

• Ensure that you know why they aren't completing the requested task.

If you ask a participant to try something specific and they seem to be aimlessly wandering around, that can indicate that they forgot. If so, a gentle "What are you trying to do right now?" can help steer them back towards focusing on what you want them to attempt.

However, aimless wandering can also indicate that a participant does remember but is completely lost in the navigation. If they do remember the desired task, try questions like "Where would you expect to find the place to look for flights?" "If you really needed to fly out tomorrow, what would you do at this point?" "Is there anything you see here that seems like it's confusing or not what you'd expect?"

Not understanding terminology can be one motivation for seemingly aimless wandering. For example, if your participants are not familiar with travel, they may see a form that requests that they enter their desired airport code. However, if they don't know what an "airport code" means, they probably aren't going to try using that to search.

• Most people are not primarily auditory learners.

I'm an auditory learner, so if you tell me something out loud, I'll often be able to remember it. However, the majority of humans are visual learners or a combination of visual and audio learners. Some humans learn primarily by actively doing something.

If you absolutely can't give participants an open-ended task or a task that directly relates to their life, they'll have a much easier time remembering information if it's written. Print a packet with the tasks, write them down (or have the participant write them down), provide a presentation or file that lists them, etc.

• Are the tasks themselves confusing?

Participants can get confused by the way you word a task. ("Does 'book a flight home' mean the place I'm living this month, or does it mean my permanent residence?" "What does booking a flight 'via Chicago' mean?" etc.

Participants can get confused by the tasks themselves. ("I don't HAVE friends in multiple cities. Why would I ever care if a plane passed through city B or city C?")

Participants can get confused by what defines 'succeeding' at the task. ("I just got to a screen that shows me all the fights that go from Seattle to San Francisco and stop in Chicago. Hooray, I'm finished!")

Before you have anyone come in for a study, make sure you run the study tasks by multiple people to make sure that they understand what you are asking, how they'd know if they were finished with the task, etc.

...

For more on all of this, I'd recommend Interviewing Users.

Also, unless you're making a product/service that's only intended for men, you'll make this researcher terribly happy if your generic user is not always referred to as "he". ;)

Answer 4

As Marjan said in the comments, it could be possible that you just stumbled upon some really bad users :) Since we have no idea of how your design works and how good the architecture is, the only thing that comes to my mind is this kind of situation:

-> The user start the action -> The user land on a page where he doesn't know what to do anymore -> Either he click on a random link or gets lost somewhere else

So, why don't you give him a headstart? When the user start searching for the flight, you could register the action and, if he doesn't take the action to the end and stumble somewhere else, you put a message on everypage the user visits, something like

"Hey, where you trying to book this flight? Go haed, click here!"

For the other questions:

How do you find the right balance for your product between 'absolutely fool-proof' and 'this takes time'?

I find this to be a very interesting article on the matter. I don't think there is a certain point where you leave the "fool-proof" point and end on "this takes time".

http://boxesandarrows.com/are-your-users-s-t-u-p-i-d/

When are users ready to actually learn something and take their time in a non hectic environment?

I think this depends on the product itself and the user needs. If a user need your product more, they will obviously be more interested and ready to take time and learn it your way.

Is there any method to put them in to a specific mood so they are more open to anticipate with a product?

I suggest you this book:

http://www.amazon.it/For-Win-Thinking-Revolutionize-Business/dp/1613630239

it has wonderful insights on the practices and patterns that the game industry takes to intensify user engagement with the products. Something you could translate into yours.

In the end, I think there is nothing wrong with the way you're conducting your tests. There are probably just some flaw you need to work more on your design. Would be nice to see it anyway :)

Answer 5

You can hardly design for people who are not willing to learn.

Bu you can design for people who have already learned something similar and are not willing to learn yours. Those people are justified in their stubborn stance.

You can design for those people by properly adhering to established user interface conventions of the platform at hand, so that those users find everything where it is expected to be, responding in the expected ways.

Those who are still not willing to learn your system are those who are unwilling to learn all similar ones, and the underlying platform itself.

Answer 6

I once watched usability testing session where participants were supposed to perform a task. One participant got sidetracked from the task several times, and the researcher had to remind him about the original goal. In the follow up discussion the participant talked about how there was so many features, and he just wanted to explore all of them.

This was his way to use this particular website. Performing a task straightforward is not something he would do in real life. Maybe he would eventually reached the goal, once he has reached the other goals, like having a good overview about everything you can do on the site. It just didn't fit for the time reserved for the research.

Your goal is to make the participant to perform a certain task. Your test subject can have different goals. The research setting is always somewhat artificial, the task might be something that the user wouldn't perform in real life, or the way they use websites might be different than you anticipated. They are not necessarily not willing to learn.

This is completely normal. You have to generalize your research goals and perform the test in restricted time, and some people will not fit this restrictions.