User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
Which rating mechanism do users find more valuable in a scenario where the user is looking for an article in a Knowledge-Base where there are multiple articles that cover a single topic?
Is a 5-star rating more user-friendly and valuable than a simple up/down vote model?
I beg to disagree with the other answers, so, I would argue the main differentiating factor is the time it costs for the user to rate. Voting up or down can be done in seconds and is especially useful if there is a clear concept of what up and what down means. This does not need to be the same, for example on Stackoverflow this is "correct answer" or "bad answer", whereas in comment sections it tends to mean "I like" and "I dislike". In general this means that one should use a star rating system if either scoring items is rewarding to the user (e.g. better recommendations) and thus worth his time, or if rating items is a fairly rare occurrence. On the other hand, if you expect users to rate countless of items in general it will be hard to convince them to use a multi tier ranking system and two tier or even one tier (Facebook "like" button) systems will work better.
If you don't differentiate the different levels clearly enough it will quickly happen users only vote 0, n-1 or n stars. This still allows you to run a lot of statistics on it (e.g. when accounting for user average weighting etc.), but it's nowhere near as good as when users distribute their votes well. For this reason implementing a multi tier system is far harder than up/down vote option. One can improve on this by adding labels to the rating system or making explicit rules on how the voting should work (see netflix.com(Deprecated, they used to show descriptions whilst hovering over stars) or tweakers.net(Dutch)) for two good examples of this. On the other hand with a two tier system there tends to be simply no right or wrong, it's all up to the user. Even on SE you can upvote only bad answers and downvote only good answers and you likely won't get banned (correct me if I am wrong, but I believe there are no wrong votes on SE).
Another issue raised in some of the articles is that with up/down the result is often displayed as a score, whereas with stars it's often shown as an average. Now, internally you will probably be using proper weighting in all cases when calculating averages, but the real crux to the problem is presentation towards the user.
The aforementioned dutch website tweakers.net for example assigns scores -1 to +3 to their rating system (-1 being flame, 0 being offtopic, +1 ontopic and +2/+3 an excellent/exemplary read) and use averages, specifically the median, to decide how to present objects (objects with a median of 2 or above will get a darker background). +3's are so rare that they do not need to worry about showing the number of users that voted etc. On the other hands, if you take a site like netflix they simply won't show average star rating till they have got enough data and even then they try to switch as quickly as possible to a personal "predicted rating" rather than showing averages. And the last two more normal options are showing the number of users voted (really common) and total scores (the sum of all star ratings, less common, but I have seen it a couple of times now where rather than stars you have something like "+3615 cookies").
As the OP is asking about a specific case let's examine it. If we're talking about a relatively dedicated group of users then they are likely to be willing to do star ratings. However! I would advice you to present this as a crowd moderation tool, rather than as a measurement of how much users like it. In other words, make the top level say "Excellent and Recommended article!" an average score "Nice read, learned more than expected", two 'stars' something like "Found my answer" and 1 'star' would say "Needs work." (of course those are just some sample labels). On the other hand, if you've a huge number of users that just hop in quickly, want their answer and leave just as quickly you're more likely to get data using just a two tier system.
Both the 5-star rating and up/down vote models have their place on the web especially to sort out the good from the bad.
However, in your case I would look at the way that stackoverflow.com functions as they have multiple answers which pertain to one single topic on each of these boards.
The reason they use an up/down vote model is because it allows users to see which of them is the most effective and you can see in more detail how many users up voted.
Where as in a 5-star rating model you take the average of all of the votes, but this does not bring into account how many users have voted. Therefore you can have a 5 star article which only have 10 votes beat out a 4 star article which may have 200 votes and may be more effective.
The up/down vote model allows users to make a more informed decision especially when comparing multiple articles that pertain to one topic. Hope this helps.
A 5-star system wholly depends on having a discriminating audience.
This may be true for your knowledge database but in the general case can cause large amounts of skew when people either only vote 0 or 5.
An up/null/down 3-level system removes the ambiguity of the meaning of the intervening stars, but really needs more voting users to produce useful results.
Do you need 'user friendly' (down voting is rarely seen as 'friendly') or do you need data value?
blipfoto.com is one example of where the 5-star system has broken down and upsets its users regularly. 5-stars has become an up-vote action, and there is no down-vote mechanism (short of 'report abuse', which would at time be valuable.
I think it really comes down to what type of data you have and how it's used. 5 stars only adds value if cases like "Good but not perfect" or "Problematic but not useless" are relatively common. A product review site naturally fits the 5 star model. I could see a knowledge base either fitting or not fitting depending on the type of content it gets populated with and user behavior.
You could initially enable a 5 star rating; but then look at the voting patterns after a reasonable amount of use. If you see all 5 values being used frequently then keep it; if most of the votes are either 5 or 1 then converting it into a simple up/down system makes sense.
I think up/down works well if multiple people can vote. Over time, the like-to-dislike ratio should be a decent representation of how "good" something is. It many ways, it's analogous to a star rating:
If only one vote can be cast, I'm strongly in favor of a 5-star rating system. A great use case to consider is a music library. Traditionally, music libraries have offered 5-star rating systems (iTunes, Winamp, etc...).
Recently I tried Google Music, and they only offer up/down, which I find woefully inadequate. If you can only categorize songs as "good" or "bad", how do you distinguish between your favorite songs versus ones you just find decent? Think about an album: there are songs you love and can listen to incessantly, songs that you like and can listen to frequently, songs that you're okay with but sick of quickly, songs you flat out dislike under any circumstances, etc... in that case, a 5-star rating system works well, because it offers much-needed finer-grained categorization.
In summary, I think that a plus/minus "naturally" turns into something approximating a 5-star system, so long as there's a decent volume of votes. If there can be only one vote cast, I think 5-star rating system is superior.
A 5-star rating is more valuable than an up/down vote model. The up/down vote model is equivalent to a rating system in which you can attribute zero or one stars, which allows for less discrimination.
Certainly for people reading the score, a scale of six (five stars plus the situation where you have zero stars) says more. People giving a score might have to think harder with a scale of six than with a scale of two, however.
I don't think I would disagree with any of the previous assessments of the differences between the systems of Multi-Level and Single-Tier rating systems. However, consider for a moment what additional information you are looking to get from the star system. If you see the 5 star system provides a level of percision, so why not a 1 through 10 rating? or better yet what % score would you give? The more accurate the system the less precise the information is you are going to get. No one thinks in such terms. HOWEVER, people also do not think in Aristotelian patterns. We often fall into these categories. Like, Love, indifferent, annoyed, angry. So why not consider a hybrid. 1 to 2 thumbs up and 1 to thumbs down for instance. Provide a balanced scale and throw people who do not vote in the indifferent or no-rating category. This m
