Displaying smoothed ETA prediction or not?

Question

I'm working on displaying the ETA (Estimated Time of Arrival) for an item delivery using a predictive model that evolves over time. The issue I'm facing involves smoothing the transition between initial and subsequent ETA predictions and how to show it to customers.

Problem Description

The model initially provides a very inaccurate ETA (the star in the figure attached represents this initial prediction). As time progresses and more information becomes available, the ETA predictions become more accurate. This can result in a large gap (20-30 minutes) between the first and second predictions. After the second prediction, the ETA often remains constant for a period before eventually updating. For instance, in the provided figure, the ETA remains at 10 minutes from 17:31 to 17:37 before starting to decrease slightly.

I am smoothing the ETA values to handle the transition from the initial inaccurate prediction to the more accurate subsequent prediction, using a simple exponentially weighted moving average. This leads to the smoothed line (red) being extremely different from the actual prediction (blue line).

Question:

Is there a user experience-based reason to adjust the model's behavior, and thus displaying the red smoothed line, or is it acceptable to display the actual values despite huge jumps and ETA value remaining the same for a few minutes at a time?

Any insights or suggestions for improving the display of ETA predictions would be greatly appreciated.

Answer 1

From what I understand, you have some "hallucinations" at the very beginning and then you get consistent metrics.

A few things to think about:

• What is the time interval between the hallucinations and the actual values? Is it a few seconds, a few minutes or hours?
• How important is for your users to have "any ETA" vs "no ETA at all"? Do they need it as soon as possible?
• How does having incorrect information affect your users? How much do they rely on that ETA?

Base on that, I have 3 answers:

1. The initial values are not critical to users, or the time interval is small enough. In this case, just ignore those hallucinations completely and just display the actual values, no smoothing. You will need to figure out a way to distinguish between hallucinations and actual values, maybe use a time delay at the beginning, or something like that.

2. The initial values are important to users, or they need data as soon as possible. In this case you do need to display the hallucinations, but you also need to add an indicator (icon, badge etc.) to notify the users that the data has a low probability. You should also add a tooltip explaining why. As soon as accurate data comes, change the indicator to a different one saying that the accuracy is now high. Maybe you can explain accuracy in percentages, if that helps your users make better decisions.

3. If you have enough money, resources and engineering experts, you could create an AI system that is capable of predicting those values and determining which are real and which are hallucinations. This would be the best solution overall, but it's quite expensive.

---

In conclusion, my suggestion is to display only the actual values, no smoothing, and, if necessary, add an accuracy meter next to the values.