Which rating system precision is better for hotel reviews: 5-star or 10-point and why? [closed]

I'm designing for online travel portal, where user can reviews hotels. But I see IMDB used "out of 10" and Google Play store used "out of 5".

Which is good for hotel reviews and why?

• I searched other OTA .like booking.com agoda.com.. They preferred out of 10. May 6, 2018 at 6:55
• Any rating system that doesn't start at 0 inherently sucks. A rating of 4 out of 5 seems to suggest that the stuff is rated at 80%, but since the minimum rating is 1, it actually means the rating is at 75%. Also, for a completely garbage item, you have to rate at least 20% with the 5-point scale. The 10-point scale is slightly better as the minimum rating that garbage gets is 10%. But it is better to just start at 0, and also provide for negative rating so that users can appropriately rate stuff that is worse than garbage. May 6, 2018 at 15:55
• @MaskedMan: See also professional movie/video game critics who rate on a 7-10 scale (and never use 1-6 at all unless the product is amazingly bad). May 6, 2018 at 19:53
• @MaskedMan Pretty harsh review of one-indexed rating systems. A five point scale shouldn't be interpreted as a linear numerical score anyway, it's just a ranking. A four star item isn't necessarily twice as good as a two star item, nor should it necessarily be interpreted as 80% or 75% of ideal. I have never heard of a rating system with negative numbers. Why not have a system that goes from -10 to 15, to cover things that are better than awesome? The choice of range is rather arbitrary, but so long as it's understood by the users, it doesn't really matter. May 7, 2018 at 13:32
• @MaskedMan Numbers can be ordinal instead of nominal. A score of 8.8 is better than a score of 4.4, but with Likert-like scales, there's no reason to believe it's twice as good. The difference between a 1 and a 3 is not necessarily the same as the difference between an 8 and a 10. You are layering mathematical expectations on a scale that's simply not designed to be used that way. May 7, 2018 at 14:32

Be as close to what users understand and mean to say.

In general, I believe the 1-5 rating scale is better than a 10 star scale. Why? Simplicity is the answer:

• I don't know where the middle in a 1-10 scale is. Is it 5 or 6?

• If it is a 0-n scale, I don't know what actually a 0 is: lowest answer or no answer? If you show it as stars, it would most likely be "no answer" as if it should be the lowest one user still would need to click something (and imagine them clicking "nothing" which would stand for 0 stars). So "no answer" becomes a natural choice for 0 and we end up with a 1-n scale anyway.

• As I cannot really tell the difference between 6 and 7 or 7 and 8 I would need to think about it.

Hence, speaking of generalities, 1-5 scale seems to be the best choice. It's simple to translate to the natural language and offers the neutral point in the middle that is easy to detect. It just goes from:

1. Very negative
2. Somewhat negative
3. Neutral
4. Somewhat positive
5. Very positive

Simple, natural. Now, one problem to solve is: If I am negative about something, then why would I give it any star at all? The thing is that actually if a scale addresses both positive and negative feedback it does not go from 1 to 5 – it goes from -2 to +2. To support this meaning you can use a different representation than stars, e.g.:

• :c Very negative
• :( Somewhat negative
• :| Neutral (yay, this smiley looks still a bit negative, give it a slightly positive meaning instead)
• :) Somewhat positive
• :D Very positive

Remember that this only regards this kind of scenario. Sometimes it may be better to show a completion of some progress for example when you would actually want to show 0-100% scale. But in most cases I believe it is like the above. Then you can go for a -1 / +1 scale, which a thumb rating actually is.

And remember, once you implement a scale it's hard to move to another one because of the rating data coherence. You need to make a good decision now. Fortunately, you can experiment, e.g. by sending emails to only some of the users, directing them to a rating system of your choice, giving them a possibility to add a written review and seeing how much they succeed or need to explain their selection in words (e.g. "I gave it a thumb up but in fact I am only partially satisfied because...").

• Is that why we sometimes can select half stars? May 6, 2018 at 13:12
• "If it is a 0-n scale, I don't know what actually a 0 is: lowest answer or no answer?" That can be easily solved by using radio buttons, or explicitly providing a checkbox with option "this stuff is complete garbage, I want to rate it as 0". The user shouldn't be forced to give even a single star for garbage. The -2 to +2 scale is a better idea. May 6, 2018 at 15:59
• I’d just like to point out that in some countries exams at school are graded from 1 (best) to 5 or 6 (worst, failure). So I’d really go for stars instead of just a number scale to make it less ambiguous. May 6, 2018 at 15:59
• The use of unhappy / happy faces here is much better than stars for actually conveying how people feel about their experience, and it's such a good idea that I expect it would be shot down immediately by one of the higher ups.
– n_b
May 6, 2018 at 17:36
• Three stars visually doesn't look neutral at all, it looks mildly positive (more than half full). May 7, 2018 at 14:34

Both 5- and 10-star rating systems invariably end up causing J-shaped distributions, making the intermediate ratings meaningless.

Thus, for usefulness of data, neither is a good solution.

This is exactly the reason both Netflix and Steam now use only thumbs-down and thumbs-up. Research has shown that the 5-star system ends up with lots of arbitrary data.

However, this decision has been highly criticized. Personally, I think a 3-rank "thumbs up, thumbs down, neutral" system would be best, but I haven't seen any research to back that.

• Possible mistake is that you mix rating of videos with rating of hotels. Linked article in one of answers addresses this difference. For videos (emotional experience), up&down. For hotels (product/service quality), stars naturally fit better. May 6, 2018 at 22:27
• And Youtube - this post includes a J-shape chart across the entire platform (from 2009). May 6, 2018 at 22:39
• J-shaped distributions are only really a problem if you mistreat the data. As long as you recognize the data as ordinal, not real-valued, and use only appropriate methods for ordinal data, you're fine. May 8, 2018 at 7:10
• Arguably, the current Netflix system is "thumbs up, thumbs down, neutral", with "neutral" being "watched, but did not rate". May 8, 2018 at 9:10

Go with a 5-star rating with possibility of half stars.

Reasons:

• this scale offers 10 grades, visually presented as 5 stars
• users who love rating at 5-star scale are not confused and can quickly post their rating
• users who find 10-grade scale more accurate, can still award for example `4.5` stars (score 9 of 10)
• it is not new to understand, because today, overall ratings are displayed in decimals (e.g. `4.4`) so users are already trained in seeing decimals
• this is not even new, I already rated something this way (I forgot where)

Design suggestion:

• A quick draft of control in

• view mode
• edit mode:
• Description: Show stars as usual, but on rating form, show left and right halves of the stars separated so it is obvious they can be touched/clicked separately. After activation, light up stars from start to activated point so the proposed rating will become obvious. Submit with separate Submit button – maybe you already have one if your form allows optional details of rating.

• For mouse/pen users, add the same highlighting effect when pointer is hovering over the stars so users get a hint.

Range:

It is on you whether you will choose possible rating range `0`~`5` stars (11 steps on scale) or `1`~`5` stars (9 steps). This depends on further context of rating system and was not part of your question. I would not start with lowest rating at `0.5`, though.

In context of hotel reviews:

I often find myself wanting to give more than four and less than five stars. For example, my hotel was great but I feel I reserve 5 stars for special experiences, not for ordinary ratings. If you expect that some of your audience exercise more sophisticated thinking (business travelers, congress/conference visitors), maybe really give them something more than only 5 grade scale. But keep it simple enough for occasional travelers.

Like/dislike rating is unsuitable.

I think that like/dislike rating is not suitable for hotels because J-shaped data distribution is not a case which typically occurs when services or products are rated although it can happen in case of sudden change of quality. Go check for example Google Play Store where statistics are visible for each app. Like/dislike is for different type of commodities and after few days I am actually a bit surprised how many people here at UX site does not recognize that.

• Very nice one. Let's just remember that in the mobile first world and case here the hover might not be the best solution to go. We would need a touch-ready alternative. May 6, 2018 at 13:47
• @DominikOslizlo – I was just thinking about exactly the same point and how to change this part of the answer May 6, 2018 at 13:48
• A slider filling in the stars as you swipe from left to right maybe? ;) May 6, 2018 at 13:49
• @DominikOslizlo – thanks for commenting. I thought of the slider, too, but it looks like showing users two scales, one above other. This might look overly complicated. I think typical experience from present apps is that users click the stars directly. So clicking plus explicit Submit button can work better, I think. This is also how existing rating forms (in various app stores) work and users already know this. I updated the answer. May 6, 2018 at 14:44
• 5-star range is good, because when presented with a 10-star rating some (maybe most) users would rate between 5-10, (giving a rating of 5 for worst items). May 7, 2018 at 10:19

10 stars (or 5 stars with the ability to give half-levels) give more precision, and allow better distinction between someone saying a place is good, very good, or extremely good.

There is a common problem with having novices give 5-star ratings, where many people end up rating places that they thought were "good" with 4 or 5 stars. The result is that the average rating for most pretty-good places in such a community end up being some decimal between 4 and 5, with the only options for votes by people who liked it 4 or 5. People then have to decide which places they like will get 4 and which 5, and may look at the current average and vote to swing them closer to 4 or 5. In other words, you get pretty crude and skewed input. I.e., for places you like, a 5-star system really only offers a choice between 2 values.

Whatever rating scale is used, I think it can help a lot to provide visible meanings to each rating, to help people vote more consistently with each other. You can see the difference in what can typically be expressed by the typical meanings available on 5-point or a 10-point scale:

``````5: Excellent
4: Good
3: OK
2: Poor
``````

On a 5-point scale, if you can only choose "Good" or "Excellent" for a positive experience, there's little room for one person to rank the places they like against each other.

``````10: Outstanding
9: Excellent
8: Very Good
7: Good
6: Better than average
5: Average
4: Below average
3: Poor
1: Awful
``````

There's a lot more room to express your actual opinion of a place, even within the good and not good places.

Adding a zero rating may also be good, both for catharsis and to detect the people who are irate or feel a place is really terrible.

For a hotel, a 5-star scale gives no ability to express the difference between:

``````A truly outstanding/special/memorable/favorite place.
A great place that did nothing wrong.
A really good place.
A good place.
``````

So, for people who care about giving an appropriate rating, having to decide to rate a hotel either a 5 (excellent) or 4 (good) tends to always present a dilemma of which way to be inaccurate.

The problem with 1-5 or 1-10 ratings is that most people's attitude toward a product encompasses multiple orthogonal issues, including (but hardly limited to)

1. How much is there to like about the product.

2. How many unresolvable reasons are there to dislike the product.

3. How hard is to resolve issues that would otherwise be reasons to dislike the product.

Someone who only uses the product a tiny bit, but has no problems whatsoever, should that person rate the product higher or lower than someone who uses the product a lot and eventually tweaked it to work flawlessly, but spent a lot of time getting there? How about someone who was never able to get the product to work without suffering serious on-going problems, but still finds it useful to do some obscure things that no other product can do?

Different users of products have different needs. A product that will require one to spend a month struggling with it before it really becomes useful, but which could be very useful after that, might be perfect for someone who would expect to use it for many years, but far less so for someone who needs to perform some task once and would never need the program again. No single 0-5 or 0-10 rating will be able to rank programs according to their ability to meet different peoples' needs.

I would suggest to use the rating system which is most easily represented in your UI. If you are using stars then it is WAYYY too much information to show 10 stars next to each item.

Notice that IMDB simply uses a fraction as the main focal point. If you actually visit their site then they very sparingly show the actual stars. Notice how it's an overabundance of information:

It would not be enjoyable to see a listing of 100 items with 10 stars next to each one.

In either case, if you ever find that you chose the wrong number then it's easy enough to convert from 5 to 10 or from 10 to 5.

If in doubt, reduce choice.

Ultimately, the rating is used to help others make a yes/no decision. As such, what works best is a simple differentiator between yes and no.

Asking yes/no questions also gets the most reliable answers. Worst case you get a "maybe", so don't offer users the choice for a "maybe" and you can only get reliable answers.

Once you increase choice to, say a five star system, reliability suffers. Turns out people map those five points differently. Some will claim there is no such thing as perfection, and thus they will never give five stars to anything. Others are of the opinion, that three stars is still "pretty good", while at least someone will be adamant that three stars or anything below five stars, really, is already awful.

You see, more choice will only result in a mess, so get rid of it if possible and just stick to "yes" and "no" if you want reliability.

This is simple. The more information communicated to the application, the more the application knows. The cognitive load and effort of using more-detailed ratings just increases linearly while the amount of information passed increases exponentially.

Looking at the act of inputting the information alone is meaningless. Every experience that results from the rating is affected by the quality of that rating.

Use a scale of one-to-ten. Rating things on a scale of one-to-ten is the least cognitive load for most people. Most of us will think in percentages and have to convert the percentage to fit in the five-point scale and will have to rethink the score if it lies near the borderline of another score (like the difference between giving four or five stars).

Use more than one rating input whenever natural. Don't make users do the work of figuring out one rating to cover multiple things. Single-rating systems like the one used by Uber tests great for first-time and new users because they don't know the experience of getting a five-star driver with a two-star car or the cognitive load of rating a great driver with a bad car.

• Rating on a scale of 1 to 10 has the problem that most people will give a score of 1, 9, or 10. People aren't actually very good at gradations beyond "like", "neutral", "dislike", and a ten-point scale just gives the illusion of detail.
– Mark
May 7, 2018 at 21:05
• @Mark U or J patterns happen in 4 and 5 point systems too so I'm unclear what your point is? A more detailed scale is more detailed so I don't know what you're trying to say with "illusion of detail"?
– moot
May 8, 2018 at 21:20
• "Illusion of detail" is when you measure something in a way that gives the appearance that there's more detail than is actually present. I could measure the height of an elephant with a micrometer and get a number with five decimal places, but that isn't actually any more accurate than "about 3.2 meters". Similarly, a ten-point scale and a five-point scale both appear to give great detail, but there's not actually much more information in them than a simple count of "like" and "dislike".
– Mark
May 8, 2018 at 23:31
• Yes I fully understand what you're trying to say when you made up the term "illusion of detail". The problem is that in math, logic, programming, information architecture, UI, psychology, perception, etc. there is far more information in 10 points than there is in 5. What documentation or testing or articles or whatever supports your new terminology and assertion?
– moot
May 9, 2018 at 4:54