How do you make users vote objectively and not from emotions?

Question

I've seen on Youtube many informative videos on massacres and natural disasters where users don't vote objectively. Rather, they vote based on their emotions, which is why these videos always have a substantial proportion of dislikes. They dislike the event that the video talks about, but probably appreciate learning about the event.

The problem with emotional voting is that it screws up the website's trending algorithm. Not as many informative videos will rise to the top because they are getting so many dislikes. (Youtube probably has some advanced algorithm to combat this, however it is not perfect). How can we change this voting UI to make users vote objectively on a video's quality, not on what they feel about the content of the video?

Answer 1

Voting will never be objective, given that people will form (differing) opinions about something more or less immediately after encountering it. Controlling this is very difficult. As an example, Reddit advices users to vote on comments based on what contributes to conversation, not based on what they like or agree with, but it's pretty evident that many will still vote based on what they like if you read through comment threads. Users will already have some preconceptions of what an arrow up or down, or a thumbs up/down, means. These things will affect how they use those UI elements.

I would avoid using the word "like" or symbols like thumbs up/down anywhere. Making guidelines for which type of content should be voted up or down could be worth it, but it depends on the community and their goal for visiting the site (StackExchange vs. YouTube, for example). I would also consider asking concrete questions like "Was this video informative?" if that's what I wanted to know. However, if someone has a negative mind-set due to disliking content in a video they just watched, stopping them from giving a negative vote is generally going to be very difficult.

Answer 2

One possible approach is to embrace the fact that the users aren't going to be completely objective while voting. And so you could try using a more explicit voting system, ie. more choices and more specific than the generic "like/dislike"-"upvote/downvote" pattern.

The perfect example for this is BuzzFeed's rating system:

---

Update:

Another alternative is trying to make a clearer distinction between objective and subjective (rating) questions [Taken from Goodfil.ms]:

---

Update:

I just found another cool rating system, this one is from Canv.as.

It has an explicit description for each type of (up)vote, so they are actually taking advantage of the user's emotions to make the rating more objective.

Answer 3

Ultimately you can't.

People vote (or don't vote) for all sorts of reasons, none of which you can control. You can publish guidance (presented in tooltips or linked to from information icons) but you can't guarantee that people will read it or interpret it in the same way you intend.

If you change the buttons to more clearly indicate you are looking for votes on the quality of the video, people might still be reluctant to vote it up or down because (as you have already surmised) they don't want their action to be interpreted by others as "liking" or "disliking" the event.

Answer 4

The problem is, as other have indicated, that a simple "like" and "dislike" is not sufficient for proper statistical information. If you did a more formal questionnaire, showed a video, and asked "Did you Like or Dislike this video", you would get very confused respondents and responses - it is not clear in either case what "like" is intended to mean.

The same applies on facebook statuses. If someone says "My cat has just died", is it reasonable to "like" it, as an indication of sympathy? Or of pleasure because the moggy is deceased? What does a "like" mean here? It is not clear to anyone.

The simple "Like/Dislike" voting system is very flawed, but it gives people a chance to provide a simple response. 90% of the time, it is fine, which is pretty good for such a simple system.

Answer 5

Your mental model of human thinking is naive, sorry. Take a look at Daniel Kahneman's work (see his book "Thinking,fast and slow") if you want to understand why you simply cannot separate "emotion" from any other aspect of human cognitive processing.

Answer 6

For those who say it's impossible: did you see slashdot's voting system?

You actually have to categorize your vote: 5, Informative or 4, Funny, or 5, Interesting.

I don't say it's perfect. I don't say it works for everyone. I just say it isn't impossible.

Grabbed a topic which has 100+ comment from today's homepage:

http://it.slashdot.org/story/12/07/26/1840241/worlds-most-powerful-x86-supercomputer-boots-up-in-germany

And to have something which brings up emotions (the concorde disaster did, I guess):

http://politics.slashdot.org/story/12/07/26/2127207/flight-4590-didnt-kill-the-concorde-costs-did

Again, I don't say it's for everyone. I just say it might be possible this way.

Answer 7

The problem you face is of conflicting interests. While you may want voting to be objective, media outlets want eyeballs on content, so use eye-catching, emotive titles to grab the users attention. This means that when users vote, they have already been heavily influenced by the title of the content, the (emotive) comments, and so on. Personally, I'd move from a simple like/dislike approach to a rating system explicitly based on what you want to capture - it's the same principle as having buttons labelled with precisely what's happening, e.g. "I agree to the terms and conditions" rather than "OK", because users will ignore most of the surrounding copy. This will not guarantee a wholly objective response, but it will make it more likely that people will respond on the thing that you want them to be objective about.

Answer 8

AS Chris pointed out, you cant.

There are always going to be conflicting emotions about a certain video or article or even a comment which was made and the number of likes/dislikes can not be used as an indicator of how acceptable it was.

You could of course tally the number of likes/dislikes and utilize the total number of views if available to try and integrate it into your trending algorithm but using the subjective "Likes" would not work due to how people react to an video of an event.

You also have to realize that you could have a video such as the highlights package of a world cup football match which might have a lot of downvotes from the people of the country who lost the finals even though there was nothing negative about the video.

Answer 9

We've had versions of this debate come up a lot at work. Someone observes that some users may vote WRONG, and proposes a solution. Then someone else realises the user could still vote WRONG. As UX designers, we shouldn't accuse the users of human error.

I think the real confusion here, hinted at by other answers, is that voting systems don't quantify user thinking because thought isn't being measured. They are quantifying behavior, counting how many people voted to "like" for example. What combination of intellect, emotion, optimism, pessimism, nostalgia, etc. occurred within the user's mind when they took the action to vote, cannot be known. Consider my Facebook friends, constantly posting politically charged material they say they hate, so they can then tell us all how much they hate it. Obviously, they love hating it. It interestes them.

Clever use of context aside, I think most voting systems really just measure interest (voting) vs apathy (not voting). It would be interesting to see a voting system that gave the votes context by setting them relative to views that drew no votes - essentially abstentions.

Answer 10

Accept that you can't have your cake and eat it. You can't capture everything about a video in just a like or dislike. So something like: 1: how did you feelt about these events? 2: Did you think this was important to know about?