Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I am working on calculating a "quality of experience" metric for an interactive application. This takes into account things like how responsive the app is, and how good the output is (sorry this is phrased somewhat abstractly, but the question is really not about the specific application). It is very easy to calculate how responsive the app or how good the output is right now (for example, in the past second).

The question: how to aggregate those instantaneous numbers into an overall quality of experience metric for a user session?

An average does not produce a very intuitive result. To see this, let us suppose the user used the application for an hour, and there were 10 individual problems (glitches, instances of bad user experience) each lasting a second. The average would be the same no matter how they were ordered in time (also very high: 99.7% of seconds are problem-free). But the experience may be quite different, if the problems

  1. occurred back-to-back in the middle of the one hour (10 seconds total), or

  2. were spread out every six minutes during the whole hour, or

  3. were spread out every minute for ten minutes

Can you please point to any research on how users rate intermittent problems depending on how they are spaced out?

share|improve this question
2  
One thing that might help is to look at these metrics from the context of individual tasks that the user is trying to complete. Remember that metrics are just there to help you get an overall idea, it doesn't replace understanding the interactions as they take place from a user's point of view. –  Michael Lai Nov 2 '13 at 5:03
1  
This is a great question. But finding the answer needs research enough for at least one Ph.D., and as far as I am aware, nobody has ever done that. I have read some of the most recent publications which take baby steps towards such metrics, and nobody uses an established method for it, so while I haven't read all the literature on the topic, it is probable that such a thing doesn't yet exist. –  Rumi P. Nov 4 '13 at 11:16
add comment

1 Answer

This is an interesting question: really what you're talking about is that badly defined thing we call 'stress'.

Too much bad happening too quickly (situation 1 on the list) and people get stressed - and when people get stressed they start making bad decisions.

You can argue that 'stress' is a state of overarousal - and then this is covered by something called the Yerkes Dodson Law

As the graph shows, a bit of arousal / stress improves performance - but for complex tasks performance decreases once arousal / stress goes too high. If you watch people in this kind of situation they actually stop dealing with the information they are getting from their environment.

And of course, it is common sense that if you throw too much at somebody too quickly they'll just get mad with it and give up.

So in terms of an overall metric I'd add a (heavy)loading factor on any issues that happen back-to-back.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.