Managing an App's Loading Time Expectations

Question

I was recently asked to come up with a framework to benchmark loading time expectations based on the tasks being performed in the app.

I found that there were three categories in which I could define how time was used by the application (tentatively named functions, tasks and processes, anyone know standard terminology about this?):

• A: (functions) any "busy" time in the application that doesn't require any feedback (like a progress bar, or mouse unavailability) to the user while performing the task. For example, saving a value on a field (which shouldn't require much time); any "instant" operation.

• B (tasks) any "busy" time in the application that requires status feedback to the user, such as a busy cursor, a progress bar, a thinking bar, etc.

• C (processes) any "busy" time that will disable the application for as long the process is happening and takes longer that what a user would wait sitting in front of the device. Like for example syncing a huge database overnight.

The image below, explains the former in terms of time and helps categorize Performance test scenarios in terms of the expected time of completion and developers best-case-scenario expectations.

I came up with this to standardize the ways we can allow our application to behave (as I couldn't find anything about it), but a) I was wondering whether someone had a better idea on how to manage loading-time expectations or how to categorize loading time as a non-functional requirement, or if someone thinks I'm missing something?

The values mentioned there are kind of arbitrary and the marks are not precise, however, I think the pattern is, and its interesting.

b) would it be better to measure the time that certain operation within our application takes and from there categorize it under a specific name or feedback type? (and if so, how do we push the application's performance further?)

Answer 1

About measuring time of the operations, that is something you sure have to do, in the debugging part, or even in development you have to do it, find bottle necks and fix them. Don't focus on miliseconds, there is no need to go that deep, but check for parts where you may have cycles, or processes that sit idle for long, or parts that process too much information and can be made lighter.

Once you have those measures in your development phase, you start feeling how well the app is working, then compare with similar ones and benchmark against. If your app doesn't have any competitors or similar ones, check for something that processes similar information or similar in weigth (information processed).

If your graph is oriented to the user, remember that information is interesting and useful, but you should also provide ways to use it, for instance, if something is talking too long, you may tell the user that the connection speed is too low and he should go to a place with better signal. If it has duplicate information, you can advise about that, etc.

You also have to analyse the logs you get from the app, and check, with real data why something is talking too long and react. The best reaction is to find the way to cut the time something takes, but sometimes is just not possible, the the best reaction is to accept the fact, and just tell the user that the operation should take that time (or that period of time). Of course you can try to split the process in smaller pieces and then the feeling for the user is that things have improved. If that's possible.

About the graph and classification you showed, it seems pretty good to my. I'll just add that the tasks and processes definitely need a system to keep the user informed about the progress.

Answer 2

Haven't cracked the numbers well yet, but I think that the framework used above to measure/identify functions is way too granular.

Loading-time expectations could be managed on the spot using visual feedback.

The image below is broken into three different feedback options. At any given time, Pointer and either, Status or Progress Bar are displayed (Progress & Status bar don't go together).

Feedback will also depend on how the operation is performed; background or foreground.

---

References:

• Progress Bars
• Mouse and Pointers

Answer 3

This is consistent with most state of the art I have seen. Your values though are not entirely correct, 1s max for A is too high, I would probably say it should be around 500ms. Ideally, you'd give feedback that the users' input was considered though, so they don' t trigger an action again.

Seems like B and C also have generous times

There is also another distinction which defines whether or not the wait time is predictable or not. Some actions are somewhat predictable (writing on disk) while some other are not. You need to differentiate the feedback between the two.