5

We are trying to build the 'Yelp of Personal Finances' at SuperMoney. It's basically a community driven reviews site in the personal finance niche.

One of the things we are doing differently is storing a lot of weighted product attribute data and combining that with user sentiment reviews to algorithmically score various products/services.

The end result is a numerical 0-100 ranking of companies.

enter image description here

I was hoping to get the community's thoughts on using a numerical rating system as opposed to a conventional star rating system. I realize it breaks convention but, personally, I feel like the star rating system is generic and imprecise. But my concern is, whether I'm sacrificing usability for what I perceive as a better system.

We are also showing a purely user rating via the star system. I'm not sure if this is confusing or not. I should probably distinguish the two better.

Would love to hear your feedback on the above and anything else on the site.

1
  • Did I understood correct: numerical rating for community thoughts and star rating for user review?
    – FrankL
    Oct 30, 2013 at 8:14

5 Answers 5

3

I think both rating methods have their places, but it is confusing for me to see them in the same interface. Would it be possible to boil the two ratings down and show them using the same kind of scale?

Percentage ratings are often confusing to me... did that user think the site was 61% good and 39% awful? If there are multiple users, does that mean that 39% of them disliked it? If it's communicated well though I think it can be really valuable: Rotten Tomatoes uses it to simply state what percentage of the reviews were favorable.

Star ratings are handy for enforcing simple standard on reviews: 5- excellent, 4- good, 3- ok, 2- poor, 1- awful. If this is adequate to convey the ratings then I say go for it, but I definitely understand how it can be an oversimplification of the information you're trying to convey.

Hope this helps!

3

Speaking as a user I have a bit of a love-hate relationship with 5-star rating systems. They have the merit that they are familiar, well understood, and with enough accurate data samples they can be a good reflection of feedback.

The 'hate' side of my relationship with them is that accuracy is pretty unlikely. In reality, if people have had a 'no problems' experience they will vote 5. Where I've had an exceptionally good experience, I've no way of showing that above the mass 'no problem' rating.

Therefore a 'Percentage' rating system sounds attractive but would probably suffer from a similar effect, and not have the benefit of simplicity. It might even discourage users if they run through a thought process of "63%? 67%? 70%? Oh, I can't be bothered."

With all rating systems I prefer to be guided in the meaning of each rating value. BoardGameGeek (a boardgame review and rating site) uses such a system. A user is guided into a 1-10 rating scale ranging from

"You won't catch me dead playing this. Clearly broken." (1)

through

"Good game. Usually willing to play" (6)

to

"Always want to play it, and don't expect this to change" (10)

Interestingly, the user is allowed to use decimals. This can be a route to your percentage rating via a back door. I'd feel (irrationally) more comfortable saying 6-and-a-bit than 63%.

Also, BGG uses a dual rating system. One based on the simple average rating and one 'moderated' by an algorithm. Users can use either to sort the data by.


PS. My first posting here. Please be gentle if I've messed up on the etiquette :-)

2

I don't think your 100 scale is abitrary as long as a user can understand the system behind. The star rating is well known by own usage, but 63 out of 100? What facts have to be realized to get it? May be a good idea is to link the 63 to table where you show how many points for what.

First I looked for relationship of stars and point ratings myself, so I assume due to the proximity one thinks of a connection. May be a "Our rating:" subtitle can solve it.

What do you think about a rating in percent? It propably matches better the financial topic and mental model of users. And every one knows its out of 100.

A different note: the red stars are very prominent for the eyes as there isn't too much red onsite. Don't know if they earn that weight.

1

From a customer point of view, you might want to be able to sort on both scoring systems. In your current implementation, you are forcing them to sort based on your website weightings score.

Your weightings may not be what the customer is after, or if they don't understand how the system works, they may ignore them entirely.

I like the idea of the "our rating" and a separate "customer rating" column. If your main goal is service through transparency, you may get better experience through better service.

Of course ratings, regardless of source, can sometimes be seen as tainted if too heavily weighted in certain directions, so separating the ratings out may provide a service where customers know you are segregating the scores as you know they can be different.

Something to test and see what comes up.

2
  • I think having multiple scores is an interesting idea. I guess we need to weigh utility vs simplicity though. I don't it to be simple and elegant. If there are too many numbers I'm worried it will add complexity. What I'm hoping is that we get very good at the scoring algorithm to provide a list that user's can trust. Nov 1, 2013 at 22:40
  • That's one of the holy grail elements of the web. Customer trust! I guess splitting your ratings out, but using different visual indicators might help. So there's a numerical score from your company, and a star rating from customers. An easy visual differentiator. Either way... good luck :) Nov 4, 2013 at 16:47
0

Trip Advisor is a good example in this case. They're balancing presenting official star ratings of hotels on the one hand, with a user-generated score on the other. Here's some aspects that I think are important for how they succeed in this:

  • Placement: The star-rating is placed underneath the hotel's name & placed beside the standard details (e.g. address, phone number). The user-rating is displayed more prominently to the side, in a larger font & associated with the reviews that follow.
  • Design/Colour: They give a visual as well as a percentage for the user-rating, but distinguish it from the official rating by using their brand colours (green, vs grey for the offical rating), and using filled circles rather than repeating the stars.

Your case is slightly different, as you're displaying a table of results with both together, but some things that might make it easier for a user to understand:

  • As SidetrackedByLife mentions, you should split out the two ratings into different columns to show they're distinct, with clearer labelling.
  • As per Trip Advisor, I would give visual prominence to your own rating and try to give it a clearer association to your brand (font, colours, etc.)
  • Have a tooltip over your site's rating which shows the breakdown of categories used.

On a side note, I'd be inclined not to use the user rating as an input when calculating your rating. By keeping them completely separate, users can weigh your advise on one hand & then choose how much to factor in customer satisfaction themselves. As the example of Rotten Tomatoes that Dennis gives, this can be a useful feature when you want to compare & contrast critics opinions with the public. If they put the public score as part of an overall, it would take away the usefulness of collating & providing two scores, in my opinion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.