# How to model the probability of an item being selected from search results

Say a user is presented with a list of 10 suppliers to choose from. The only information shown is the name of the supplier. The list is sorted by a score that is calculated from various data (volume, reputation, locality etc) so that the first item is the one we have decided is the 'best' according to the scoring system.

Is there a way to model the probability of each item being selected? In the absence of any other information, will the user always choose the first item?

Ideally I'm looking for links to research, not anecdotal evidence.

• Would the user know why the supplier are ranked this way? Is the user given the scoring system? Commented May 5, 2017 at 12:22
• No, the scoring algorithm is not fixed and may change over time according to the distribution of hits.
– GC.
Commented May 5, 2017 at 12:57

## 1 Answer

Is there a way to model the probability of each item being selected?

To begin with we may associate your suppliers list to a search result page. There are many studies about this topic, older and most famous ones I think are those published from Enquiro (2007) and Chitika (2010 and 2013). Note that many other studies have been published in later years.

Let's pick, for example, Chtitika:

As you can see the first position has almost 35% of the hits and after 5th position each one has less than 5%.

Later studies show a similar distribution and somehow similar numbers, with the notable exception of Slingshot (2011) and Catalyst (2013) (where trend is similar but distribution is slightly more uniform for the first five positions).

Note that your situation is somehow different. First of all because search engines will favor the best match in their ranking then we all learned that there are better chances to find what we need in the very first positions.

Internet queries are, also, different from your specific case (sometimes they're categorized in navigational, transactional and informational) then you can't really blindly apply mentioned results. Studies made analyzing AD clicks are maybe more meaningful in your case. Please read their published values for a better analysis.

In the absence of any other information, will the user always choose the first item?

Choosing for a supplier, an user will be attracted by the first item in the list (see for example The first one wins: Distilling the primacy effect) but there are many other factors that come playing here (known name, rating from other users, price and everything else related to each one). There are many studies about recommender systems because they're extremely useful to rank products you may be interested in (and then to buy them...)

Unfortunately each case is slightly different then you should really search for literature about your specific field but you may use this (older) studies as starting point:

• G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17, 2005
• P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-N recommendation tasks. In ACM Conference on Recommender Systems, 2010
• In the case of Google, I don't think we can assume people clicked the first item because it's first. They might have clicked on it because it's actually the best result. Or one of the best. Google, via some complex magic, is pretty good at figuring out what I really want. Commented May 5, 2017 at 13:42
• Ken, I agree, I mentioned that: we learned to trust the search engine. For search results we see both factors combined Commented May 5, 2017 at 15:39