I am working on calculating a "quality of experience" metric for an interactive application. This takes into account things like how responsive the app is, and how good the output is (sorry this is phrased somewhat abstractly, but the question is really not about the specific application). It is very easy to calculate how responsive the app or how good the output is right now (for example, in the past second).
The question: how to aggregate those instantaneous numbers into an overall quality of experience metric for a user session?
An average does not produce a very intuitive result. To see this, let us suppose the user used the application for an hour, and there were 10 individual problems (glitches, instances of bad user experience) each lasting a second. The average would be the same no matter how they were ordered in time (also very high: 99.7% of seconds are problem-free). But the experience may be quite different, if the problems
occurred back-to-back in the middle of the one hour (10 seconds total), or
were spread out every six minutes during the whole hour, or
were spread out every minute for ten minutes
Can you please point to any research on how users rate intermittent problems depending on how they are spaced out?