Haven given a lot of thought as to which site would be the best fit for this question, I decided it was ultimately a user experience one. Feel free to suggest a better site if you think it would fit better elsewhere.

The paper, From Amateurs to Connoisseurs: Modeling the Evolution of User Expertise through Online Reviews heavily focuses on data mining and customer reviews. It's main argument is that users with a similar level of experience on a subject, tend to give similar ratings. For example, someone who is new to drinking alcohol may not like whiskey if they've never had alcohol before, but after growing accustom to it, they may find very enjoyable.

This made me wonder, what is the purpose of ratings? Why is it Amazon has 5 stars that customers can choose and IMDB has 10 stars a person can rate. For example both Citizen Cane and Starwars the force awakens have 8.4/10 stars on IMDB. What information is this actually trying to convey to the user? It seems like comparing apples to oranges to say that means they are exactly the same level of "goodness".

On Amazon it wouldn't make sense to conclude that because a toaster had 4/5 stars and a hairdryer had 3/5, then it's better to get the toaster.

In conclusion, I am concerned that such rating systems lead the way to people trying to force a statistical pattern, where none actually exists.

  • I remember reading something about YouTube changing from a 5 Star system to a Thumbs Up/Down system because almost all ratings were 1,3 or 5 stars. Here's something similar: techcrunch.com/2009/09/22/… I think it's down to how the rating needs to be interpreted. – Brett Jan 20 '16 at 5:03
  • 1
    @DarBrett that's another point, I bet one person's idea of a 7/10 is different than another person's idea of a 7/10. For example one may rate something 7/10 and another 8/10, all though they both enjoyed it to the same degree. – Celeritas Jan 20 '16 at 5:04
  • Also YouTubes rating system kinda functions as a controversy rating since people will downvote videos because they're politically maligned to something in the video rather than just rating based on the quality of the video. With as Star or numeric system you'd just get a highly controversial video rated as 'average'. – Brett Jan 20 '16 at 5:28

You are correct that star ratings are limited, but they are still useful as a rough guide of quality. On Amazon, I can tell which products I should probably avoid, and which are worth investigating more from the number of stars. On IMDB, I can tell whether a movie is likely to be broadly good or terrible.

It's true that I can't gather more fine-grained information from them. However, this is a limitation of aggregated opinion data, not a problem with the star rating itself. Capturing data about people's opinions and aggregating it into a quickly usable measure is a beneficial exercize, but the result will always be limited by the subjectivity of the initial input.

Star ratings work best when they are only one of the tools available. Take Amazon for example: the overall star rating is only one piece of information they provide out of many. In addition, they also offer:

  • The number and distribution of ratings
  • Actual text reviews.
  • Information about reviewers (such as whether they actually purchased the product and what other reviews they have written).

In the overall Amazon system, the star rating is just a quick initial way to gauge quality, and once you have focused on a few products, you can delve into more detailed information if you want to know more. Overall, this results in a highly effective tool for comparing products.


Why stars in amazon :

In amazon it's just a way to assure the user whether the product is god or bad. A star rating on the higher side ( say above 3 with a large number of ratings ) can give a belief to the user that the item has been used by many people and is decent enough to use. One person's 3 and other person's 3 might be different , but still they can conclude that product has optimum quality.

The toaster - hair dryer example

What you said is correct , one cannot compare a hair dryer and toaster using star rating. But a user can compare two different brand hair dryer/ toaster using the star rating and star ratings can have a massive effect on choosing between two kinds of similar products.

Same goes with IMDB :

Shawshank Redemption has the highest IMDB rating ever - doesn't mean that it is the best movie ever made. Just like the above 3 metric in amazon , say an above 7 assures that the movie is decent.

I guess that is what a star rating is used for. On places where you can't go binary - star rating helps you choose good, bad or better.

Software review sites such as capterra also uses star rating system because of the same reason. To give the user a sense of good and bad.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.