There's a search box, powered by Apache Lucene, that user enters terms into to find records that they need, (eg. They're likely to be searching for people, or addresses, or ID numbers, etc).
John Smith John OR Bob Name:John Country:Australia Name:(John OR Bob) Name:[Andy TO Charlie] Name:"Mary Jane"
Lucene allows you to add a fuzzy modifier to terms, to find similar terms.
John~ //by default the fuzzy value is 0.5, lower value means more fuzzy. John~0.1 //Might return James, Joan, Jon etc John~0.8 //Might return Jon
I've been tasked to make all queries fuzzy by default, so for a simple query, when the user enters 'John Smith' he gets results for 'Jon Smith' etc.
The way to achieve this is to parse and change the query
John Smith to
If the user needs to turn the fuzzy search off, they can specify the fuzzy term
The issue is, this gets a bit more complicated for the less standard queries.
For example should:
Name:(John OR Bob) evaluate to
Name:(John~ OR Bob~)
John NOT Jon ->
John~ NOT Jon~
The issue with the
John NOT Jon query is that when Jon is fuzzified all those results are excluded, this query is likely to not return any results.
One option is to say 'Making results fuzzy by default isn't a good idea, you should just train the users on how to use the fuzzy modifier'.
Another option is to come up with a series of conditions for when a term should be fuzzyified and when it shouldn't.
Fuzzy any standalone terms. Don't fuzzy anything in square brackets. Don't fuzzy anything in double quotes. Don't fuzzy anything following a NOT. Fuzzy stand alone terms in round brackets.
The problem I see with this, is that it might make things confusing for the user, as to why sometimes a 'John' term returns 'Jon' and why other times it doesn't.
From the initial problem (we want to make basic searches fuzzy) we have actually a kind of complicated problem.
I'm wondering what the best set of rules or conditions here is, such that it's not needlessly complicated or hard to use for the user.