UX Pattern for indicating information that might be wrong

Question

I’m working for a startup that crawls data from news articles and displays it in a table. For example: “Bayern Munich won 2:0 against BVB. Both goals were scored by Lewandowski. Müller assisted both goals.”

1. Result: 2:0
2. Winning Team: Bayern Munich
3. Goals scored by: Müller

As you might have noticed, the goals were not scored by Müller. The crawling algorithms do make mistakes. For this reason, we are looking for a UI/UX solution, that communicates that information might be wrong. At the moment users do get the feeling, that the extracted data are facts (100% safe) - we want to change that.

There are some approaches:

1. All customers (B2B) do get an onboarding anyway. They get instructed, that information can be wrong. No additional highlighting in the UI necessary.

2. We are labeling all information in a certain way. Google, for example, is showing a grey “traffic-light” (see screenshot) when estimating my location.

3. Do you have any other ideas or are you familiar with some UI patterns that might fit my problem?

Answer 1

It is not uncommon for one to put a question mark in parenthesis at the end of a statement to indicate that it may not be correct. At least in American English.

One might say for example, "I think this is the right approach(?)" when they are both stating this as a fact, yet also questioning it at the same time.

So for your example, the results becomes...

1. Result: 2:0
2. Winning Team: Bayern Munich
3. Goals scored by: Müller(?)

Answer 2

Typically when there's a chance that information may have been interpreted incorrectly by an automated system, the user should have an opportunity to easily view the original text from which the 'facts' were sourced.

You see this in automatic translation systems (see google web page translation below), where not only is the original text displayed, but the user is given an opportunity to either flag a bad interpretation or contribute a better one. Thus allowing you to improve your algorithms.

Similarly in other natural language processing, the facts that have been pulled out from documents (even in the same language) are commonly backed up by a link to the relevant highlighted section of the original document, in order for users to see the snippet in context of the original document and surrounding text, and thus make their own judgement.

The ability to 'see the original' (if they wish) not only helps to check facts, but boosts user's confidence due to being able to access the trusted original evidence. This means they are not being forced to accept the extracted information as shown. As with any 'black box' system, users tend to distrust something they don't understand.

The above all assumes that any of the information might be wrong, as opposed to having determined that only some of the information falls below some threshold of certainty in its correctness - e.g. an ambiguity threshold. If you have such a threshold then you probably want to highlight where the greater ambiguity begins and ends. The problem with highlighting is that it emphasises, thus bringing more attention to content that is less likely to be correct. For that reason, you might want to consider de-emphasising such text relative to its surroundings.

Answer 3

I worked on a very similar thing where a search algorithm would trawl a database to look for citations mentioned in document:

The search engine would scan a document and where it found a citation go back to a database and give it a reliability rating from 1-3. 1 being fine, 2 being dubious, 3 being out of date and returned all the results to a table with a traffic light system. This is what you should use.

It relies on how good the search engine is. The more rigorous the more it can tell you what's wrong with the source document.

At the very least have a traffic light system. Something like:

What you need: the engine to return a list which is easily scannable for the user to pick up the state of documents