How to analyze quality of data to be used in an autosuggest or autocomplete?

Question

How does one review and analyze the underlying data set for an autosuggest in order to know what mitigating choices to take with the autosuggest UX? I am concerned there may be problems in a data set eroding the user experience.

In this case, it's a tagging solution using a list of company names. Users will be able to create a new tag if there is no match, so if the autosuggest is not effective we may have both an abundance of redundant tags where the user assumed the company was not already in the system, and untagged entries where the user became frustrated and chose not to tag.

For years there has been a informal set of data in the voice of the users - "Disney" vs. "Walt Disney Company, The" but we're using data from another source - we'll use "starts with" on each word of the company name to mitigate this specific issue, but are there any tips on reviewing the data to determine other choices we can make before going live?

I've read with great interest many of the Q&As around Autocomplete and Autosuggest, and I'm grateful for the UX tips and perspectives I have already found here.

Answer 1

Tagging, when in the context of an autocomplete textbox where users can write virtually anything, is analogous to a search problem. You not only have to worry about when there are multiple accepted forms of a tag (eg. Disney v. Walt Disney), but also possible spelling errors.

Without going into too much detail (search is a complex topic and I'm not sure what the optimal complexity/usefulness ratio is for your tagging issue is), there are a few things you can do to "clean" the text before you do your searching:

• Case folding: Make both input and output lowercase or uppercase (doesn't matter which, so long as it's consistent). That way string comparisons for walt disney vs. Walt Disney are possible.
• Punctuation: For your purposes, Disney, Walt is no different than just simply Disney Walt. Yes the words are in the wrong order, but that's a separate issue. Strip out most or all punctuation, just in case.
• Stop word removal: there are a lot of words in the English language that aren't very useful when searching text, like the, an, and of. Combined with the above, a straight string comparison for Walt Disney, The would match walt disney with no extra manipulation.
• "Bag of Words": In your example above, Disney vs. Walt Disney still would not match even using startswith since the latter includes Walt at the beginning. If word order isn't that big of an issue (it is when you're searching for sentences, but less so for tagging since tags tend to be short single or pairs of words), you might try splitting the input into words and sorting lexagraphically before comparing.
  You might even want to compare word-by-word after sorting to make a count of how many words in the input match existing tags.
• For spelling correction, Levenshtein Distance is a simple and effective way of detecting misspelled words. Set a small tolerance on the number of acceptable "edits" and you may have more accurate results.

In terms of literature, a good place to start would be Intro to Information Retrieval although any source that focuses on search and ranking might prove useful.

Answer 2

There are different options, and "starts with", as you mention, is not a bad one, although a simple but is, names that start with "the", or even common words that are not really helpful to identify a company unless you use the complete name or almost complete.

Since the process is going to be implemented in something as "informal" as tagging, you should focus on the common or "street" names given to those companies. In the end, is more important that the system comes up with a good approximation. Of course you can use a system like stackexchange, and instead of presenting only one suggestion, you can offer a few, plus the invisible retaging of synonyms, which makes things easier for the client.