Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

This question already has an answer here:

Which rating mechanism do users find more valuable in a scenario where the user is looking for an article in a Knowledge-Base where there are multiple articles that cover a single topic?

Is a 5-star rating more user-friendly and valuable than a simple up/down vote model?

share|improve this question

marked as duplicate by Danny Varod, Charles Wesley, Evil Closet Monkey, Matt Obee, Code Maverick Jun 11 at 18:53

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

3  
Personally, I think star-based rating systems make the UI look rather cheap. But opinions aside, I can place more trust in a certain article when I see it has some upvotes, while an article with 5 stars could've been rated by just one person. –  VinceCgto Jun 10 at 14:45
3  
    
See my answer here: ux.stackexchange.com/a/23025/687 –  Danny Varod Jun 10 at 16:08
    
Depends on how it's then used. If you're browsing through some list of media, say, what does '1.2k people upvoted this' mean to you? How many downvoted? How many abstained? It's meaningless without that context - which is why Google Play Store uses /5* reviews to rate instead of the +1s (used for social). –  Ollie Ford Jun 10 at 21:11
    
@VinceCgto If you see a answer on a StackExchange site with 1 upvote, did it have just one vote, or were there 100 up and 99 down? With star ratings, you can usually see that X number of people rated and the average rating. –  David Wilkins Jun 11 at 16:17

7 Answers 7

up vote 18 down vote accepted

Time & Simplicity

I beg to disagree with the other answers, so, I would argue the main differentiating factor is the time it costs for the user to rate. Voting up or down can be done in seconds and is especially useful if there is a clear concept of what up and what down means. This does not need to be the same, for example on stackoverflow this is "correct answer" or "bad answer", whereas in comment sections it tends to mean "I like" and "I dislike". In general this means that one should use a star rating system if either scoring items is rewarding to the user (e.g. better recommendations) and thus worth his time, or if rating items is a fairly rare occurrence. On the other hand, if you expect users to rate countless of items in general it will be hard to convince them to use a multi tier ranking system and two tier or even one tier (facebook "like" button) systems will work better.

The danger of multi tier systems

If you don't differentiate the different levels clearly enough it will quickly happen users only vote 0, n-1 or n stars. This still allows you to run a lot of statistics on it (e.g. whn accounting for user average weighting etc.), but it's nowhere near as good as when users distribute their votes well. For this reason implementing a multi tier system is far harder than up/down vote option. One can improve on this by adding labels to the rating system or making explicit rules on how the voting should work (see netflix.com or tweakers.net(Dutch)) for two good examples of this. On the other hand with a two tier system there tends to be simply no right or wrong, it's all up to the user. Even on SE you can upvote only bad answers and downvote only good answers and you likely won't get banned (correct me if I am wrong, but I believe there are no wrong votes on SE).

Score vs Averages

Another issue raised in some of the articles is that with up/down the result is often displayed as a score, whereas with stars it's often shown as an average. Now, internally you will probably be using proper weighting in all cases when calculating averages, but the real crux to the problem is prestation towards the user.

The aforementioned dutch website tweakers.net for example assigns scores -1 to +3 to their rating system (-1 being flame, 0 being offtopic, +1 ontopic and +3 an excellent read) and use averages, specifically the median, to decide how to present objects (objects with a median of 2 or above will get a darker background). +3's are so rare that they do not need to worry about showing the number of users that voted etc. On the other hands, if you take a site like netflix they simply won't show average star rating till they have got enough data and even then they try to switch as quickly as possible to a personal "predicted rating" rather than showing averages. And the last two more normal options are showing the number of users voted (really common) and total scores (the sum of all star ratings, less common, but I have seen it a couple of times now where rather than stars you have something like "+3615 cookies").

Specific case: Knowledge-Base

As the OP is asking about a specific case let's examine it. If we're talking about a relatively dedicated group of users then they are likely to be willing to do star ratings. However! I would advice you to present this as a crowd moderation tool, rather than as a measurement of how much users like it. In other words, make the top level say "Excellent and Recommended article!" an average score "Nice read, learned more than expected", two 'stars' something like "Found my answer" and 1 'star' would say "Needs work." (of course those are just some sample labels). On the other hand, if you've a huge number of users that just hop in quickly, want their answer and leave just as quickly you're more likely to get data using just a two tier system.

share|improve this answer
    
A lot will depend, I think, upon whether a particular would be motivated to rate everything, or only rate things which are especially good or especially bad. I think participation might be better if there were both "mild like" and "enthusiastic praise" buttons, with voters encouraged to cast a "mild like" for anything they felt had some real value, while using "enthusiastic praise" sparingly [the weight of a voter's "enthusiastic praise" could be weighted according to the voter's reputation and how sparingly such praise was given]. –  supercat Jun 10 at 16:51
    
SE does analyze votes, if you upvote the same person too many times in one day, after some time it will roll back your votes. (same with downvotes). –  Mooing Duck Jun 10 at 22:28
    
@MooingDuck: Yeah, but that's because of the assumption that in that case you are breaking the multiple accounts rule and thus constitutes fraudulent voting. –  David Mulder Jun 10 at 22:29
    
Great answer, great examples. What it comes down to, multi-tier rating systems tend to be quite more valuable than one/two tier rating systems, that is, if they're implemented properly. That's exactly the hard part. –  VinceCgto Jun 11 at 6:20
    
+1 for the point about giving each rating level an explicit and useful meaning (e.g "Found my answer" vs. "Good"). Of course, this applies regardless of which system is used. –  Cheeseminer Jun 11 at 8:45

Both the 5-star rating and up/down vote models have their place on the web especially to sort out the good from the bad.

However, in your case I would look at the way that stackoverflow.com functions as they have multiple answers which pertain to one single topic on each of these boards.

The reason they use an up/down vote model is because it allows users to see which of them is the most effective and you can see in more detail how many users up voted.

Where as in a 5-star rating model you take the average of all of the votes, but this does not bring into account how many users have voted. Therefore you can have a 5 star article which only have 10 votes beat out a 4 star article which may have 200 votes and may be more effective.

The up/down vote model allows users to make a more informed decision especially when comparing multiple articles that pertain to one topic. Hope this helps.

share|improve this answer
    
Thanks! This helps a lot. –  Aaron Benjamin Jun 10 at 14:49
    
Glad to help! :) –  Nick_M Jun 10 at 14:53
3  
Actually, you don't have to sort by average rating when using 5-star rating: evanmiller.org/how-not-to-sort-by-average-rating.html –  RedSirius Jun 10 at 15:37
    
...and continuing what @RedSirius said, you shouldn't use an average rating for comparing items in a 5-star system. –  zzzzBov Jun 10 at 20:11
2  
The problem I find with the Stack Overflow model is that its very hard to compare like for like... outdated answers never really get downvoted when they cease to refer to newer best-practice approaches. Also, really good answers sometimes get a quick tick and don't get many up-votes. That can mean when casually browsing it is hard to actually weigh up how good an answer is without testing it. I've seen factually wrong answers with 60 upvotes, and some excellent answers with only 1. Average answers that have been around for ages can have 1000s of upvotes. Not convinced that's best UX. –  Matt Coubrough Jun 11 at 0:57

I think it really comes down to what type of data you have and how it's used. 5 stars only adds value if cases like "Good but not perfect" or "Problematic but not useless" are relatively common. A product review site naturally fits the 5 star model. I could see a knowledge base either fitting or not fitting depending on the type of content it gets populated with and user behavior.

You could initially enable a 5 star rating; but then look at the voting patterns after a reasonable amount of use. If you see all 5 values being used frequently then keep it; if most of the votes are either 5 or 1 then converting it into a simple up/down system makes sense.

share|improve this answer

A 5-star system wholly depends on having a discriminating audience.

This may be true for your knowledge database but in the general case can cause large amounts of skew when people either only vote 0 or 5.

An up/null/down 3-level system removes the ambiguity of the meaning of the intervening stars, but really needs more voting users to produce useful results.

Do you need 'user friendly' (down voting is rarely seen as 'friendly') or do you need data value?

blipfoto.com is one example of where the 5-star system has broken down and upsets its users regularly. 5-stars has become an up-vote action, and there is no down-vote mechanism (short of 'report abuse', which would at time be valuable.

share|improve this answer

I think up/down works well if multiple people can vote. Over time, the like-to-dislike ratio should be a decent representation of how "good" something is. It many ways, it's analogous to a star rating:

  • High thumbs up ratio is similar to a 5-star rating
  • High thumbs down ratio is similar to a 1-star rating
  • Mixture of the two is similar to a 3-star rating
  • etc...

If only one vote can be cast, I'm strongly in favor of a 5-star rating system. A great use case to consider is a music library. Traditionally, music libraries have offered 5-star rating systems (iTunes, Winamp, etc...).

Recently I tried Google Music, and they only offer up/down, which I find woefully inadequate. If you can only categorize songs as "good" or "bad", how do you distinguish between your favorite songs versus ones you just find decent? Think about an album: there are songs you love and can listen to incessantly, songs that you like and can listen to frequently, songs that you're okay with but sick of quickly, songs you flat out dislike under any circumstances, etc... in that case, a 5-star rating system works well, because it offers much-needed finer-grained categorization.

In summary, I think that a plus/minus "naturally" turns into something approximating a 5-star system, so long as there's a decent volume of votes. If there can be only one vote cast, I think 5-star rating system is superior.

share|improve this answer

A 5-star rating is more valuable than an up/down vote model. The up/down vote model is equivalent to a rating system in which you can attribute zero or one stars, which allows for less discrimination.

Certainly for people reading the score, a scale of six (five stars plus the situation where you have zero stars) says more. People giving a score might have to think harder with a scale of six than with a scale of two, however.

share|improve this answer
    
You said a 5-star model is more valuable to users, but requires more work and allows for more discrimination. Would you still recommend using it with that being said? –  Aaron Benjamin Jun 10 at 14:51
2  
I think it depends how important discrimination is in the context. The ability to distinguish between four stars and five stars works best when there is a subjective context worth evaluating and recording. If your are concerned about telling users "9 out of 12 people found this helpful" then the stars won't be appropriate. –  Mishax Jun 10 at 14:56
    
thanks for clarifying! –  Aaron Benjamin Jun 10 at 15:08

I don't think I would disagree with any of the previous assessments of the differences between the systems of Multi-Level and Single-Tier rating systems. However, consider for a moment what additional information you are looking to get from the star system. If you see the 5 star system provides a level of percision, so why not a 1 through 10 rating? or better yet what % score would you give? The more accurate the system the less precise the information is you are going to get. No one thinks in such terms. HOWEVER, people also do not think in Aristotelian patterns. We often fall into these categories. Like, Love, indifferent, annoyed, angry. So why not consider a hybrid. 1 to 2 thumbs up and 1 to thumbs down for instance. Provide a balanced scale and throw people who do not vote in the indifferent or no-rating category. This m

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.