Studies showing the impact of response time variability on user experience

Question

Have there been any studies, or results, focusing on the impact of variability in latency of response times on the overall user experience?

I'm aware of the research on the "generally accepted" recommendations (100ms for instant, progress indicators for 1s+, etc.). However, what I'm curious to know is specifically the perceived performance difference (if any) between cases like:

1. consistent 500ms response time for all interactions
2. average 300ms response time with +/- 200ms variability

Is (1) preferable to (2), vice versa? If either, in all cases? Etc. Of course, 500ms and 300+/-200ms are arbitrary stand-in values.

Answer 1

Do you want to feel that the application you're using is generally slow or slow for the actions that one could expect to take a long time to execute..?

I would like to compare it to two different real world examples.

• If the bus is always late, you can adjust your schedule/timing so that you're right on time even though it's always late. You as a user of the bus is unaffected, because you can adjust.

However, the bus runs its own schedule, unaffected by any input from the commuters. So even though it could work in a bus scenario, it doesn't apply to a user interaction scenario.

Instead think of it as what it is, a request and a response.

• If you order food from a restaurant, do you want the quick dishes to be prepared and served in the time they take to make? Or do you want the time to be consistent for all dishes, ie the quickest dishes are served in the same time as the advanced dishes? If you ordered a plate of fries, do you want it to take the same time as if you ordered a sea basket?

Sure, if you're a party you would prefer if everyone gets served at the same time, but that has more to do with a code of conduct and is not very applicable here. If you order something fast by yourself you want that damn food to be served when it's done, not to be delayed as if someone you were with ordered the most advanced dish on the menu.

I would argue a user of an application is the same, at least I am. I realize a customized search for items takes time to perform, and that's OK. However, when I view an item in the search and then go back to the search results I don't want to wait the same time again as it took for the initial search. As a user I would feel, -Hell, there's no need for this to take this long?! Why is it doing this to me?!

So keep execution times to the actual time an individual action takes to execute, no need for unnecessary waiting.

Answer 2

Jakob Nielsen, a usability guru, states in his book Usability Engineering (1993) that response time limits is an important factor of usability. An excerpt of chapter five of the Usability Engineering book is published on his web site and answers your question very well. Blockquote from the article Response Time Limits:

Q: "You mention many times that response time is important, and there are tons of tools to measure response time, but what is an acceptable web based application's response time? What is a user's tolerance, not for a shopping experience, but for an interactive application?"

A: I wish we could eradicate the term "web-based application" because it distracts from the real issue, which is one of application UI design. We don't have special guidelines for applications implemented in C++ relative to apps implemented in Visual Basic. The fundamental usability recommendations are the same, no matter the implementation, since we are discussing user experience, not coding.

Therefore, the response time guidelines for web-based applications are the same as for all other applications. These guidelines have been the same for 37 years now, so they are also not likely to change with whatever implementation technology comes next.

0.1 second: Limit for users feeling that they are directly manipulating objects in the UI. For example, this is the limit from the time the user selects a column in a table until that column should highlight or otherwise give feedback that it's selected. Ideally, this would also be the response time for sorting the column - if so, users would feel that they are sorting the table.

1 second: Limit for users feeling that they are freely navigating the command space without having to unduly wait for the computer. A delay of 0.2-1.0 seconds does mean that users notice the delay and thus feel the computer is "working" on the command, as opposed to having the command be a direct effect of the users' actions. Example: If sorting a table according to the selected column can't be done in 0.1 seconds, it certainly has to be done in 1 second, or users will feel that the UI is sluggish and will lose the sense of "flow" in performing their task. For delays of more than 1 second, indicate to the user that the computer is working on the problem, for example by changing the shape of the cursor.

10 seconds: Limit for users keeping their attention on the task. Anything slower than 10 seconds needs a percent-done indicator as well as a clearly signposted way for the user to interrupt the operation. Assume that users will need to reorient themselves when they return to the UI after a delay of more than 10 seconds. Delays of longer than 10 seconds are only acceptable during natural breaks in the user's work, for example when switching tasks.

These time limits work well for guidance of user experience on any application. To me "Sacrificing user experience for higher performance" is not an issue, since user experience is dependent of higher performance.