3

Say a user is presented with a list of 10 suppliers to choose from. The only information shown is the name of the supplier. The list is sorted by a score that is calculated from various data (volume, reputation, locality etc) so that the first item is the one we have decided is the 'best' according to the scoring system.

Is there a way to model the probability of each item being selected? In the absence of any other information, will the user always choose the first item?

Ideally I'm looking for links to research, not anecdotal evidence.

  • Would the user know why the supplier are ranked this way? Is the user given the scoring system? – Dimitra Miha May 5 '17 at 12:22
  • No, the scoring algorithm is not fixed and may change over time according to the distribution of hits. – GC. May 5 '17 at 12:57
3

Is there a way to model the probability of each item being selected?

To begin with we may associate your suppliers list to a search result page. There are many studies about this topic, older and most famous ones I think are those published from Enquiro (2007) and Chitika (2010 and 2013). Note that many other studies have been published in later years.

Let's pick, for example, Chtitika: Percentage of traffic by Google result position, by Chitika

As you can see the first position has almost 35% of the hits and after 5th position each one has less than 5%.

Later studies show a similar distribution and somehow similar numbers, with the notable exception of Slingshot (2011) and Catalyst (2013) (where trend is similar but distribution is slightly more uniform for the first five positions).

Note that your situation is somehow different. First of all because search engines will favor the best match in their ranking then we all learned that there are better chances to find what we need in the very first positions.

Internet queries are, also, different from your specific case (sometimes they're categorized in navigational, transactional and informational) then you can't really blindly apply mentioned results. Studies made analyzing AD clicks are maybe more meaningful in your case. Please read their published values for a better analysis.

In the absence of any other information, will the user always choose the first item?

Choosing for a supplier, an user will be attracted by the first item in the list (see for example The first one wins: Distilling the primacy effect) but there are many other factors that come playing here (known name, rating from other users, price and everything else related to each one). There are many studies about recommender systems because they're extremely useful to rank products you may be interested in (and then to buy them...)

Unfortunately each case is slightly different then you should really search for literature about your specific field but you may use this (older) studies as starting point:

  • G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17, 2005
  • P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-N recommendation tasks. In ACM Conference on Recommender Systems, 2010
  • In the case of Google, I don't think we can assume people clicked the first item because it's first. They might have clicked on it because it's actually the best result. Or one of the best. Google, via some complex magic, is pretty good at figuring out what I really want. – Ken Mohnkern May 5 '17 at 13:42
  • Ken, I agree, I mentioned that: we learned to trust the search engine. For search results we see both factors combined – Adriano Repetti May 5 '17 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.