Which rating system precision is better for hotel reviews: 5-star or 10-point and why? [closed]

Question

I'm designing for online travel portal, where user can reviews hotels. But I see IMDB used "out of 10" and Google Play store used "out of 5".

Which is good for hotel reviews and why?

Answer 1

Be as close to what users understand and mean to say.

In general, I believe the 1-5 rating scale is better than a 10 star scale. Why? Simplicity is the answer:

• I don't know where the middle in a 1-10 scale is. Is it 5 or 6?

• If it is a 0-n scale, I don't know what actually a 0 is: lowest answer or no answer? If you show it as stars, it would most likely be "no answer" as if it should be the lowest one user still would need to click something (and imagine them clicking "nothing" which would stand for 0 stars). So "no answer" becomes a natural choice for 0 and we end up with a 1-n scale anyway.

• As I cannot really tell the difference between 6 and 7 or 7 and 8 I would need to think about it.

Hence, speaking of generalities, 1-5 scale seems to be the best choice. It's simple to translate to the natural language and offers the neutral point in the middle that is easy to detect. It just goes from:

1. Very negative
2. Somewhat negative
3. Neutral
4. Somewhat positive
5. Very positive

Simple, natural. Now, one problem to solve is: If I am negative about something, then why would I give it any star at all? The thing is that actually if a scale addresses both positive and negative feedback it does not go from 1 to 5 – it goes from -2 to +2. To support this meaning you can use a different representation than stars, e.g.:

• :c Very negative
• :( Somewhat negative
• :| Neutral (yay, this smiley looks still a bit negative, give it a slightly positive meaning instead)
• :) Somewhat positive
• :D Very positive

Remember that this only regards this kind of scenario. Sometimes it may be better to show a completion of some progress for example when you would actually want to show 0-100% scale. But in most cases I believe it is like the above. Then you can go for a -1 / +1 scale, which a thumb rating actually is.

And remember, once you implement a scale it's hard to move to another one because of the rating data coherence. You need to make a good decision now. Fortunately, you can experiment, e.g. by sending emails to only some of the users, directing them to a rating system of your choice, giving them a possibility to add a written review and seeing how much they succeed or need to explain their selection in words (e.g. "I gave it a thumb up but in fact I am only partially satisfied because...").

Answer 2

Both 5- and 10-star rating systems invariably end up causing J-shaped distributions, making the intermediate ratings meaningless.

Thus, for usefulness of data, neither is a good solution.

This is exactly the reason both Netflix and Steam now use only thumbs-down and thumbs-up. Research has shown that the 5-star system ends up with lots of arbitrary data.

However, this decision has been highly criticized. Personally, I think a 3-rank "thumbs up, thumbs down, neutral" system would be best, but I haven't seen any research to back that.

Answer 3

Go with a 5-star rating with possibility of half stars.

Reasons:

• this scale offers 10 grades, visually presented as 5 stars
  • users who love rating at 5-star scale are not confused and can quickly post their rating
  • users who find 10-grade scale more accurate, can still award for example 4.5 stars (score 9 of 10)
• it is not new to understand, because today, overall ratings are displayed in decimals (e.g. 4.4) so users are already trained in seeing decimals
• this is not even new, I already rated something this way (I forgot where)

Design suggestion:

• A quick draft of control in

  • view mode
    
  • edit mode:
    

• Description: Show stars as usual, but on rating form, show left and right halves of the stars separated so it is obvious they can be touched/clicked separately. After activation, light up stars from start to activated point so the proposed rating will become obvious. Submit with separate Submit button – maybe you already have one if your form allows optional details of rating.

  • For mouse/pen users, add the same highlighting effect when pointer is hovering over the stars so users get a hint.

Range:

It is on you whether you will choose possible rating range 0~5 stars (11 steps on scale) or 1~5 stars (9 steps). This depends on further context of rating system and was not part of your question. I would not start with lowest rating at 0.5, though.

In context of hotel reviews:

I often find myself wanting to give more than four and less than five stars. For example, my hotel was great but I feel I reserve 5 stars for special experiences, not for ordinary ratings. If you expect that some of your audience exercise more sophisticated thinking (business travelers, congress/conference visitors), maybe really give them something more than only 5 grade scale. But keep it simple enough for occasional travelers.

Like/dislike rating is unsuitable.

I think that like/dislike rating is not suitable for hotels because J-shaped data distribution is not a case which typically occurs when services or products are rated although it can happen in case of sudden change of quality. Go check for example Google Play Store where statistics are visible for each app. Like/dislike is for different type of commodities and after few days I am actually a bit surprised how many people here at UX site does not recognize that.

Answer 4

10 stars (or 5 stars with the ability to give half-levels) give more precision, and allow better distinction between someone saying a place is good, very good, or extremely good.

There is a common problem with having novices give 5-star ratings, where many people end up rating places that they thought were "good" with 4 or 5 stars. The result is that the average rating for most pretty-good places in such a community end up being some decimal between 4 and 5, with the only options for votes by people who liked it 4 or 5. People then have to decide which places they like will get 4 and which 5, and may look at the current average and vote to swing them closer to 4 or 5. In other words, you get pretty crude and skewed input. I.e., for places you like, a 5-star system really only offers a choice between 2 values.

Whatever rating scale is used, I think it can help a lot to provide visible meanings to each rating, to help people vote more consistently with each other. You can see the difference in what can typically be expressed by the typical meanings available on 5-point or a 10-point scale:

5: Excellent
4: Good
3: OK
2: Poor
1: Bad

On a 5-point scale, if you can only choose "Good" or "Excellent" for a positive experience, there's little room for one person to rank the places they like against each other.

10: Outstanding
9: Excellent
8: Very Good
7: Good
6: Better than average
5: Average
4: Below average
3: Poor
2: Bad
1: Awful

There's a lot more room to express your actual opinion of a place, even within the good and not good places.

Adding a zero rating may also be good, both for catharsis and to detect the people who are irate or feel a place is really terrible.

For a hotel, a 5-star scale gives no ability to express the difference between:

A truly outstanding/special/memorable/favorite place.
A great place that did nothing wrong.
A really good place.
A good place.

So, for people who care about giving an appropriate rating, having to decide to rate a hotel either a 5 (excellent) or 4 (good) tends to always present a dilemma of which way to be inaccurate.

Answer 5

The problem with 1-5 or 1-10 ratings is that most people's attitude toward a product encompasses multiple orthogonal issues, including (but hardly limited to)

1. How much is there to like about the product.

2. How many unresolvable reasons are there to dislike the product.

3. How hard is to resolve issues that would otherwise be reasons to dislike the product.

Someone who only uses the product a tiny bit, but has no problems whatsoever, should that person rate the product higher or lower than someone who uses the product a lot and eventually tweaked it to work flawlessly, but spent a lot of time getting there? How about someone who was never able to get the product to work without suffering serious on-going problems, but still finds it useful to do some obscure things that no other product can do?

Different users of products have different needs. A product that will require one to spend a month struggling with it before it really becomes useful, but which could be very useful after that, might be perfect for someone who would expect to use it for many years, but far less so for someone who needs to perform some task once and would never need the program again. No single 0-5 or 0-10 rating will be able to rank programs according to their ability to meet different peoples' needs.

Answer 6

I would suggest to use the rating system which is most easily represented in your UI. If you are using stars then it is WAYYY too much information to show 10 stars next to each item.

Notice that IMDB simply uses a fraction as the main focal point. If you actually visit their site then they very sparingly show the actual stars. Notice how it's an overabundance of information:

It would not be enjoyable to see a listing of 100 items with 10 stars next to each one.

In either case, if you ever find that you chose the wrong number then it's easy enough to convert from 5 to 10 or from 10 to 5.

Answer 7

If in doubt, reduce choice.

Ultimately, the rating is used to help others make a yes/no decision. As such, what works best is a simple differentiator between yes and no.

Asking yes/no questions also gets the most reliable answers. Worst case you get a "maybe", so don't offer users the choice for a "maybe" and you can only get reliable answers.

Once you increase choice to, say a five star system, reliability suffers. Turns out people map those five points differently. Some will claim there is no such thing as perfection, and thus they will never give five stars to anything. Others are of the opinion, that three stars is still "pretty good", while at least someone will be adamant that three stars or anything below five stars, really, is already awful.

You see, more choice will only result in a mess, so get rid of it if possible and just stick to "yes" and "no" if you want reliability.

Answer 8

This is simple. The more information communicated to the application, the more the application knows. The cognitive load and effort of using more-detailed ratings just increases linearly while the amount of information passed increases exponentially.

Looking at the act of inputting the information alone is meaningless. Every experience that results from the rating is affected by the quality of that rating.

Use a scale of one-to-ten. Rating things on a scale of one-to-ten is the least cognitive load for most people. Most of us will think in percentages and have to convert the percentage to fit in the five-point scale and will have to rethink the score if it lies near the borderline of another score (like the difference between giving four or five stars).

Use more than one rating input whenever natural. Don't make users do the work of figuring out one rating to cover multiple things. Single-rating systems like the one used by Uber tests great for first-time and new users because they don't know the experience of getting a five-star driver with a two-star car or the cognitive load of rating a great driver with a bad car.