Implications and knock on effects of late scheduled emails to the user experience?

Question

We are building a large module onto our software that allows hourly data to be processed, and sent out as emails to customers, based on a pre-determined schedule they set themselves.

Typically our user base require this data within the hour, in order to make executive decisions.

What implications would late scheduled emails have on the overall user experience, my (obvious) initial thoughts were:

• Lack of trust in the system.
• Reduction in use, in an email capacity.
• Reduced customer satisfaction

A current example:

(If an email was scheduled to arrive in a user's inbox at 13:30, and then arrived at 14:10)

Answer 1

Your initial thoughts could be correct, but is there a way for you to test it and survey users? It's the least you could do when you are facing a problem that you cannot fix.

You could ask questions about the system satisfaction of the users.

If you want to tackle this in a serious way, for a commercial product, you could consider the Quis 7.0 questions. But note that it requires a license to use it. These questions are designed to assess users' subjective system satisfaction, and include questions about system reliability and speed, and system satisfaction in general. But also questions to control for previous experiences with the system or similar systems. I can't share specific questions here since you need to accept their license terms, but you can find it here:

http://lap.umd.edu/quis/

Another important thing (but that is merely an expectation from my side) is that users forgive you if the problem occurs just seldom. But again, I guess the way to find that out is by surveying users.

I hope this helps.

Answer 2

It depends on the user expectations. e.g. if it is clear that "Processing will start at time X and may take up to 60 minutes.", then they can set the schedule and get email when they expect it.

What is probably more important is to be very clear what data is included e.g. Google Analytic graphs show a massive drop off for most recent data, with a little footnote to say 'data for last 2 calendar days not available'. Then why offer to graph the non-existent data?

Thus allow user to set schedule in terms of data included. If user needs data between 8am-10am and they expect processing will take up to 75 minutes, then an email should arrive before 11:16am

UI can reflect this quite simply

^{download bmml source – Wireframes created with Balsamiq Mockups}