I was thinking of websites like YouTube, or IMDB, which implement rating systems. On YouTube, a video can be rated by up votes, or down votes, and videos are rated by the ratio of up and down votes. On IMDB, you can rate a movie on a 1 to 10 scale, and the rating is an average.
My first question is how relevant are these ratings to show how good a material is (whether it's a book, or a movie, or a video, or a song)? I think most rating systems suffer from being very subjective. Here are some examples in which the ratings don't reflect at all the quality of the material:
- The message of the material is somewhat controversial. Consumers will give a higher rating if they agree with the message, or a lower rating if they don't.
- The material is not targeted to a particular consumer. For example, someone with no knowledge in a specific domain consumes a material that is targeted to someone who already has some knowledge. As a result, that consumer will give a low rating because he didn't understand it.
- People have different tastes. Someone may like a genre of music, and dislike another genre. Same is with movies. This creates a big problem with the rating because it no longer tells how good that material is, but instead, how popular it is, or how many of the people who consumed it liked it. This is subjective.
So the second question is: what makes a rating system good? Does objectivity make it good, and is it possible to have a rating system that is objective, and truly reflects the quality of the rated material?