6

We supply a large number of products for purchase through our web-site. There is a new initiative to apply a third-party AI product to hijack searches to return products based on both the search term and predictions from browsing history and other people's search history with successfully processed sales. If I now search for a keyword, I will get a different set of products returned than if someone else searches for the same thing. If I pass an URL of my search to a friend to compare the products, we will have different lists, so cannot discuss. The list, I find, changes day-to-day, on my own machine due to the AI's suggestions.

Is this a design "no-no"?

Should the AI be solely used for recommendations and not for the core search results?

Is there any guideline to cite that makes suggestions on this?

I have also put this on: Software Recommendations

  • 2
    What is the use case goal in situations where you would want to compare search results with your friend? – locationunknown Sep 30 at 13:04
  • 2
    @locationunknown For instance: I list all the themed short breaks and ask him to choose. I send the url: www.mywebsite.com/search/themed-short-breaks, but we are viewing different items on the list. Also, if I search and see an item I like, then search again the next day, it won't be there. – cloudsafe Sep 30 at 13:32
  • Always when I search for something will be the same or will change like change for other people? If I search today and back next month and make the same search the result will be different? – BraDev Oct 1 at 8:23
  • @BraDev The products and availability don't change but the results change greatly in position within short times depending on what other people are looking at. If I browse the internet, my browsing history will be used to change the search results, also. – cloudsafe Oct 1 at 8:29
  • I think it depends on if the search is for a broad category like "UX books" versus a search for a specific item like "The Design of Everyday Things". I would be ok if the former gave me different results to browse from time to time, but the latter should always have at least one result that's constant. Deciding what type of query it is might be a good challenge for a machine learning algorithm. – Nathan Rabe Oct 1 at 17:10
1
+50

There are 2 types of search: finding a specific answer or item or exploratory search i.e. discover the space of possibles.

For the 1st type of search, where there is only one good answer, results shouldn't change based on who the user is and what they did. This applies when you're looking for a specific information (the actors in a movie, the date of a battle, etc).

The 2nd type of search is most common on e-commerce: you have a rough idea of what you want and you search to know the different products that match your criteria. From your description, it seems your website search is closer to this 2nd type of search.

In this case, having search results tailored to a specific user based on their history and other users' actions will likely deliver more relevant results on top and thus increase conversion. Google already refines their results based on user location, past search history and activity. On Amazon, the list of results is different depending on whether you're logged in or not, because they take into account your past browsing and buying history and wish list.

You raise 2 issues with AI-tailored search results:

1- a user can't simply share a search url and expect their friend to see the same results.

2- a user might not find the same products in their next search.

You could mitigate these issues by providing a favorite / list system. Users could add one or multiple products to a list (buttons such as "save for later", "favorite", etc) and share the list with their friends.

This introduces more complexity, but you will obtain valuable information from the list users compile (with the ability to refine future recommendations based on listed items or even send tailored mailing that include listed products).

The UI element to save an item will serve as a signal to the user that this list is temporary and they should save an item if they want to find it again later.

3

I think the idea of custom search results awesome. But you hardly avoid if the user can't back to your site and make the same search and have the same result.

You need to think about how your user we feel when this happens. Do you think the user will think:

  • "Oh, maybe they change the search logic, let me try another keyword alternative" OR
  • "Oh no, they don't have the product anymore, let me Google it to find in another place"

Other points to thinking about:

  1. How frequently your users share the result of a search using a link? If they doing it, why?
  2. How do you can use this AI to improve the results equal for all?
  3. Maybe you could use it to show on top of the common search results some suggested products.

Anyway, my vote is:

YES, the search result link should keep the same product if your user shares the link.

  • I am of the same opinion but need citations from reliable sources on development standards before I upset the apple cart. – cloudsafe Oct 1 at 14:43
  • Keep in mind: If the user searched the first time and didn't convert, maybe the "AI" has an idea to get better conversions the second time around. IME, data science models, given sufficient training data, often make better recommendations than humans or simple business rules. – plainclothes Oct 10 at 2:22
0

Just a keyword search is not enough unless it is a long and very specific keyword. Filters will be required to keep the search results in order. In addition to all these, the ability to sort them in a specific order. The search engine will need to be designed to consider and prioritise specific parameters from the index in displaying results.

Considering that the indexing time is 4 hours, the same search keyword and filters should not display different results within that period as the data-set remains unchanged. If the results are varying within that period, then the search just needs to be more rigid.

AI results will generally be different for two different people since it is based on individual activities. Search results are not expected to be overwritten by AI suggestions.

  • I agree but need a citation from a standards authority to put before the CTO. – cloudsafe Oct 1 at 14:44
0

Keep in mind that the way you train the AI and configure the parameters for the search algorithm is still based on user requirements, so there is definitely nothing to say what the search result should look like - this should be based on what the user expectations are.

If the expectation of the user is that the 'search pool' is constantly changing, then for a given keyword the results should always return the latest set of results (e.g. if you add new products). This is the simplest case.

If the expectation of the user is that the search algorithm collects some information and use it to refine the search result, then for a given keyword the results can vary even if the 'search pool' remains the same. This is also a relatively simple case.

However, if you combine the two possibilities then you end up with a number of different permutations depending on what the user expectation is for each aspect of the search process. Without understanding more about the existing design or the users, I would say that you should take a conservative approach and not assume what the user expectations might be. It will be easier to enhance features compared to designing something complex and having to scale it back.

0

You will never always return the same results

This is important to remember. For various reasons, the SRP for any given terms is usually a slowly changing experience. The merchandise and inventory change, so the results should change. Maybe not today or tomorrow. But at some point, things are dropped and things are added.

So, if it's not an immutable list of constants, what level of change will your users tolerate. Chances are, more than you think.

The age of data science and machine learning

I tested an algorithmic SRP ranker nearly 10 years ago. It was pretty early for data science applications in mid-level e-comm applications, but the human intervention model was too "emotional". If we had a big day with lots of data (a few hundred thousand hits), the model might return something new the next day. It worked. Conversion rates saw a double digit lift.

10 years on, ML is everywhere. Digital products respond to people's needs. If the model is good (and remember, humans do write these models), then it will get things right more than it gets things wrong.

Users now expect their digital world to respond to their behavior and needs.

Incremental improvement

On the topic of search ranking methods, this data science article makes a keen observation:

When you reach a scale where incremental gains in relevance are more important than the rest of the factors, consider moving to an ML model. But do make sure you have a good answer to the “Optimization Metric” problem before you start working on a ML model.

The operative phrase there is "incremental gains". You need a lot of data coupled with a well-tuned model to slowly improve the performance (engagement, conversion, average order, etc). IOW, the results are not erratic.

Too big for human orchestration

And just to further emphasize a critical point with AI / ML: you are operating on massive data sets. As pointed out in this Forbes article, man + machine can do more.

… machine learning provides significant value by processing vast quantities of data to find patterns, drawing conclusions no human is capable of and enabling incredibly precise, data-driven activations for marketers.

Bottom line

If you have a large catalog, you probably have a lot of users, and even more data. ML-driven SRPs are likely to prove fruitful.

0

There is no definite yes or no answer to the question.

It depends on several factors, the same site can have very precise search and algorithm (AI/ML) assisted search.

Real world examples:

  • When I search for an "Audi Q3" keyword in a car sale portal, I would expect to see results that contain ONLY Audi Q3s, sorted preferably by price or any other sorting criteria that I can filter. But when I search for "SUV" or "4X4", then the algorithm can assist the search.

  • When I search for "expanded cork insulation" I should see exactly that, it's a product that you sell, you either have it, or not.

  • The algorithm can assist when the search returns no results, or when results may contain too many irrelevant items. Algorithms would work well with abstract keywords such as "gift ideas" in a large marketplace (Etcy or eBay).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.