Live search response time

Question

I'm looking for research around acceptable response time for a live search feature that I can provide to my back-end team.

Some definitions:

• live search: results are automatically refined as the user types
• traditional search: the user types a full query and then explicitly invokes the search function (e.g. by pressing a button)

Of course, the ideal response time for live search is less than 100ms, and that's what you typically see in a consumer-oriented public web app. If I knew for sure I could get that kind of performance out of a back-end, I wouldn't even bother asking the question.

Because I'm working with a B2B app backed by legacy systems that have been in production for decades, <100ms is pretty much out of the question for now. So I'm trying to determine what's the maximum response time that would make live search feasible? At 200ms I know you lose the illusion that the system is responding instantaneously, but is that slow enough to interfere with the feedback loop of typing and getting results, so that live search either becomes unusable or so frustrating that users tend to prefer a traditional search? What about 400ms? What's the limit? Can we go all the way up to 1000ms?

I don't think a single target number is sufficient. The response time is not going to always be exactly the same, but will probably fit into a beta distribution.

If I specify the average response time, it could be that 3/4 of responses are well within the desired range the other 1/4 fan out far beyond that number. On the other hand, if I specify the maximum response time one long running query out of a thousand could spoil what is otherwise a successful test.

Hence, I intend to give the back-end team two performance parameters:

average response time: <250ms
95% of responses: <1000ms

I'm looking for any research or information that would help me refine those numbers and back them up with more than a hunch.

Answer 1

To recap, in the article you mentioned, Website Response Times it is mentioned:

The 3 response-time limits are the same today as when I wrote about them in 1993 (based on 40-year-old research by human factors pioneers):

• 0.1 seconds gives the feeling of instantaneous response — that is, the outcome feels like it was caused by the user, not the computer. This level of responsiveness is essential to support the feeling of direct manipulation (direct manipulation is one of the key GUI techniques to increase user engagement and control;
• 1 second keeps the user's flow of thought seamless. Users can sense a delay, and thus know the computer is generating the outcome, but they still feel in control of the overall experience and that they're moving freely rather than waiting on the computer. This degree of responsiveness is needed for good navigation. (...)

Miller 1968; Card et al. 1991:

"0.1 second is about the limit for having the user feel that the system is reacting instantaneously … 1.0 second is about the limit for the user’s flow of thought to stay uninterrupted … "

Resources:

Nielsen, J. (1993). Response times: the three important limits. http://www.nngroup.com/articles/response-times-3-important-limits/

Nielsen, J. (1997). The need for speed. http://www.nngroup.com/articles/the-need-for-speed/

Nielsen, J. (2010). Website response times. http://www.nngroup.com/articles/website-response-times/

So, for search, people expect a response time of 100 ms or less and the suggestions shouldn't appear after more than one second.

What I think you should consider is the fact a live search, that display results can be combined sometimes with auto-complete, so it should be a difference and it is no need to worry if the maximum time of response is not under 400ms or 900ms. For this, you may found interesting this question and the accepted response.

I have a filter form for a list. One of the fields is a text search field. When a user types in the field, I wait until the user has stopped typing for a half second (500 milliseconds), and then perform the search. (...) A delay of one full second (1000 milliseconds) seems too long to me, but rather than pick an arbitrary value, I was wondering if there was some sort of consensus for the length of the delay.

My conclusion is 1 second should be the maximum.

Answer 2

I too had a similar specification problem for a software project that I'm working on. The development team wanted a specification for how fast the results had to be returned after the user typed (or deleted) a character. This is not how fast the auto-complete strings were displayed, but the actual results of the search.

I approached the problem slightly differently. My UX heuristic (that I defined myself) was that the results should be returned before the user could type the next character of their search string. This meant I had to estimate how fast the user base would type:

• 60 wpm top end (especially when we've had our coffee!), derived from:
  • an average typing speed is ~35-40 wpm, see: https://en.wikipedia.org/wiki/Words_per_minute
  • the user base is engineering and technical staff, which use computers for a significant portion of the day, and I'm assuming are therefore a bit more practiced at typing
  • measuring myself, I come in around 50 wpm
• 5 characters per word avg (in the English language), see: http://norvig.com/mayzner.html

This works out to 300 characters per minute (not counting spaces), which is 5 characters per second. This results in 200ms as an acceptable response time.

In your case, you could tweak the parameters of this model to fit your user base. I believe this approach would give you a good value for the average response time in your beta distribution specification methodology.