6

I'm in a situation where there is no accounts. There is only offer-centers (which need quality ratings) and offer-takers. Every time an offer is taken a transaction is taken there is evidence of it, it's possible to have per-transaction reviews.

There is some cost to faking a transaction, but having a good rating is probably worth more.

The question is pretty multi-faceted. The core question is "how do I do reviews well". It breaks down into questions like "How do I deal with fake reviews, positive and negative" and what examplifies trustworthiness?

Links to descriptions of rating systems like eBay, Reddit's, Amazon, etc, are valuable answers too.

(Note: I don't think any single answer can be best, but I hope people will offer perspectives on reviews)

Improve this question
5
  • This is a complicated question, because even big companies (a great example is Amazon) have the same issue with "real" reviews. Now having a real person write it as apposed to a bot, that's a different thing. You could use a Captcha or other ways of validating the person writing the review.
    – UXerUIer
    Aug 8, 2014 at 15:09
  • 1
    To clarify further, what I mean by a "real" reviews is whether the person writing it is actually being truthful or not. That's where it gets tricky, and Amazon has that very same problem, but they don't care about it. It's more about the user base whether they can figure out who's being truthful or not. And maybe, if that were the case, then you could probably get a system set up where the user base can report the user's reviews?
    – UXerUIer
    Aug 8, 2014 at 15:12
  • 1
    Reporting user reviews could definitely help. I'd prefer doing it though the "useful" voting system, spam is typically not very useful. But yes, there seems to be no good way invented yet.
    – Lodewijk
    Aug 8, 2014 at 15:29
  • How about a way of giving a user a unique token they can reference when submitting reviews? So instead of creating an account, they get a token that they can input every time they interact with the site? At this point, I'm just suggesting anything that comes to mind, putting security aside (because there could be terrible outcomes if someone steals your token).
    – UXerUIer
    Aug 8, 2014 at 18:42
  • The primary reason for account-less-ness is reducing correlation between transactions. There's other reasons, a slight protest against accounts, frictionless first orders, etc. Maybe I can deterministically hash IP addresses (or another unique ID) with something to salt it properly, to deduplicate and merger. But it's a cheat reg the original question.
    – Lodewijk
    Aug 10, 2014 at 11:25

2 Answers 2

Reset to default
1

I think the best way to validate a review is to go through the process of writing one yourself, because then you'll know the kind of detail people go into when they have no idea about the actual product/service versus when you do. Most of the flaws or telltale signs of fake reviews come from the fact that people don't have a very real or accurate context for the rating and the good or bad feedback that they provide.

However, as with all types of data, the power comes from the strength and reliability in numbers, not analyzing individual reviews because it is always possible to have some false positives or negatives. There are also some very good paid review writers out there that also have actual experience with the products, but who may provide biased opinions.

I think there is a study on Chicago hotel reviews by Cornell University that is often quoted, but it doesn't go a lot into the details of how the study was conducted and what the exact findings were (or perhaps I need to dig further). There is another quoted study by Harvard university that is more recent but again I don't think you can draw a lot of solid conclusions around it.

Improve this answer
3
  • Although relevant and interesting musings, it's not much in the way of an answer. I think qualitative reviews are probably better than quantitative. But ultimately reviews are about trust. Do I trust? Did others rightly trust? Reputation free deferred trust is a totally enigmatic device.
    – Lodewijk
    Sep 8, 2014 at 1:33
  • @Lodewijk I guess I am making the point that the best way to validate a review is to get a real person to read it :p
    – Michael Lai ♦
    Sep 8, 2014 at 11:10
  • Yeah. I was thinking of having IP addresses vote for "useful" or "not useful", then let people just read the most useful thumbs up, and the most useful thumbs down reviews. Then you can judge for yourself, but others already kinda filtered for you.
    – Lodewijk
    Sep 8, 2014 at 23:30
0

What you are looking for is a "trust" or "reputation" system. You are looking for a network of interactions and evidence that a user's opinion is of value.

So:

  1. User posts a review
  2. Review gets a "weight" based on the poster "reputation"
  3. Others vote review up/down - review "weight" alter based on up/down voters "reputation"
  4. Reviewers reputation gets altered by the average score on reviews, how long they've been on the network etc

Now you do have an initial "seed" problem. The trick there is to establish an initial pool of moderators you give a high starting "reputation" to. They will then manually review initial entries using their judgement for the review authenticity. Over time this will build an ever expanding pool of people with good reputation who are able to rank reviews.

In a way this is what stack exchange does...and in fact stack exchange is a good model to consider. The principle difference between the SE system and mine is the additional weight of votes from already trusted people.

Improve this answer
2
  • Reading your post again - you are account-less so can't determine the authenticity of the taker of the transaction? And artificially inflating ratings by taking your own transactions would be cost effective? In this scenario the economics of the model make abuse of any review / rating system almost inevitable...and detecting it mighty difficult. You might be able to apply some statistics or other analysis and catch out the non-sophisticated but it all depends on the rewards on offer.
    – Andy Boura
    Aug 8, 2014 at 16:34
  • Yes, it would seem to be exceedingly hard to prevent abuse. In your system a sybil attack would still work, as up/down voters may also be sysbils. Mostly because of "Trust rot", the initially good group of mods will water down. An exploiter has the most profit from being trusted. I think what protects us more is that people just don't abuse systems so much. Which is... strange!
    – Lodewijk
    Aug 10, 2014 at 11:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.