4

Satisfaction surveys increasingly use emoticons, i.e. a sad frownie, neutral face and happy smiley. Some additionally use traffic-light color coding (red to green). Let’s asume there are good reasons in a scenario to use 5 instead of the usual 3 (or fewer) levels, much like in a classic Likert scale (which surveys by approval) or a semantic differential rating scale.

Is there a well tested (maybe even conventional or standardized) set of five emojis found in Unicode that

  1. aligns naturally (across languages and cultures) on a qualitative scale from sad to happy when used together (relative context) and
  2. has every symbol (across popular / OS-provided fonts and image sets) distinctive enough to be not misidentified when used alone (absolute context)?

I see that Emojiscore, for instance, uses 😄, 😊, 😐, 😟, 😩 (top to bottom). I would have chosen a slightly different set intuitively, e.g. 😡, ☹️, 😐, ☺️, 😍 (left to right). The linked Emojipedia articles show alternative renditions and a recent study examines how some of them are interpreted very differently.

Please note that mood surveys, like Facebook’s response additions to the Like button, are a slightly different topic: Readers select one out of a predefined set of categorical icons to represent their reaction, which usually cannot be put together on a linear scale.

Mood survey using emoticons

Related questions

  • 1
    It seems like emojis are far more humanized than, let's say, a 5 star system. Most people don't understand what a 4, or a 2 is. People can related to: faces and emotions, because they experience those on a daily basis. A "point system" seems so detached from our human behavior. Really cool question though, I'm curious to see what people answer. – Majo0od Aug 18 '16 at 12:02
  • Imho, a clean, aseptic method is better. I get what youmean, but on the otherhand I think Emojis transmit a feeling that will influence the scale with unreliable results. However, if you decide to go through, then at least make all emoji the same color to decrease additional psychological impact (such as color, of course) – Devin Aug 18 '16 at 15:54
  • Beware that they are some cultural differences regarding emojis – asiegf Oct 7 '16 at 11:34
  • LimeSurvey has a slider widget which shows a single image next to it based upon the selected value (1–5). The custom pictures most closely resemble 😭/😢, 😟/🙁/☹, 😐, 🙂/☺, 😁/😄/😆/😊. – Crissov Mar 21 '17 at 12:26
1

Proper Likert scales need 5 blocks and often more depending on how much nuance you'd like to capture, whether you allow a neutral point, etc. If you've only 3 blocks, you might as well label them "yes" "maybe" and "no". Likert scales work precisely because they don't have defined intermediate values, they're just a continuum between 2 definite values.

So when you use smilies/emojis or any other kind of intermediate labeling, you've pretty much defeated the purpose of using a Likert scale unless you're working with a population that has trouble with abstractions.

EDIT:

I let my personal terminology get into my response, so I have to explain a little.

Strictly speaking, a "Likert scale" is a continuum with 5 points on it, ranging from "completely agree" through "somewhat agree", "neither agree nor disagree", and "somewhat disagree" to "completely disagree", or other words to those effects. Each box is labeled.

Osgood's Semantic Differential scale, on the other hand, and again speaking strictly, is a continuum with some number of boxes on it, ranging from "Industrious" to "Lazy" (or some other similarly opposing pair). None of the intermediate boxes are labeled, only the end points.

Nominally, an Osgood scale is used to capture connotation, just as a Likert scale is nominally only for agreement/disagreement.

But as should be evident, both the Likert and Osgood scales are really hooking the same mammalian ability to perceive intermediate distinctions. Conventionally their domains of use are different, but that's only convention--they're pretty much interchangeable with the Osgood scale being less restrictive.

I call them both "Likert scales" because that's easier to say than "semantic differential scale" and I can never remember Osgood's name. I should probably call them "fuzzy" scales instead, since Lotfi Zadeh's work has popularised (fsvo "popular") that more generic term.

For any case more complex than the original Likert "agree"..."disagree" continuum, it's quite easy to produce a labeled scale that appears to, but doesn't, capture the information you want.

An annoying example of that is the common one used for brick-and-mortar stores such as Lowes or Home Depot. They usually have a question "How often do you visit your local Fubar Inc?" with the choices being something like "Every day", "Twice a week", "Once a month".... Such a scale can't capture "I practically live there if I'm doing a project but otherwise never". But an unlabeled scale (Osgood scale) at least has a fighting chance because the responder can pick a point between "every day" and "never" without worrying about mapping between perception and label.

For properly-chosen continua it doesn't matter much whether the intermediate steps are labeled as long as the labels themselves make sense in context. But if they don't (the Fubar Inc example), then unlabeled is more likely to produce a good result because respondents aren't forced to map their non-verbal perceptions onto fixed labels.

(Two years ago I did a bilingual survey in aid of keeping our local post office from being closed. I used a 11-grain Osgood scale and nobody, including Portuguese immigrants with little English who got a chuckle out of my broken Portuguese, had the slightest trouble deciding which box to X to indicate how important the post office is to them, how often they visit, proportion of business vs personal use, etc.)

  • Interesting point re: degrees of two values vs five distinct values. Can you edit to add any citations or further reading? – Michael Hogan Sep 25 '16 at 1:54
  • I'm retired and no longer have ready access to the literature, but if you can wait a bit, I'll see what I can track down for you. – MMacD Sep 26 '16 at 20:52
  • Thank you. No rush. The StackExchange community values answers with references, so it would be nice but not necessary. – Michael Hogan Sep 27 '16 at 13:33
  • Annoyingly, but not surprisingly given the political conditions, what little information there is remains out of my reach unless I want to pay for it. I've expanded my answer somewhat, but if it still doesn't seem valuable tell me and I'll just purge it. – MMacD Oct 6 '16 at 12:04
  • I appreciate the answer, but it’s not really answering the question. It’s “just” saying one shouldn’t label intermediate choices. An Osgood scale would require only 2 emojis, 1 for each end, but it’s still not obvious which ones one should choose. For a proper Likert scale, one would also need 2 symbols, 1 for agreement or approval and 1 for the opposite, e.g. 👍/👎, 🙆/🙅, 👌/✋, ⭕️/❌, 💯/⛔️, ✅/❎, 🆗/🆖, 😍/😡, 😃/😠, 😻/😾. However, I actually included “Likert” mostly for its default 5 steps which is more than in most smiley response sets. – Crissov Oct 23 '16 at 19:37
0

Since you are dealing with Emoji you will need to consider the technical requirements of render, the liscencing, and aesthetics of the icon set you choose. Definately don't rely on the default OS rendered characters, use your own images!

For instance, EmojiOne is free and creative commons liscenced, which is great, but you need to decide if the look is appropriate for the subjects you are surveying about. I would be concerned if these emoji were used across a wide range of dynamic questions.

The emotional weight of these pictographs go beyond a clear symbolic meaning like letters or numerals. Meaning the options are not equidistant from each other, and have no clear quantifiable value in the respondents mind. Also, the visual characteristics of (for instance) 😍 may unexpectedly skew results away from that option, when they would have otherwise clicked a "5".

Also, you have to consider the layers of meaning implied by presence or absence skin color of the emoji.

In addition, I would only consider this a "gimmick" for certain apps and use cases that are exceptionally focused on technologically progressive, mobile-first, users.

A useful experiment would be to run the same survey questions across a broad audience and compare a 1-5 and 5 emoji scale and look for statistical anomalies in the results compared.

  • Yeah, the question is basically whether there’s any documented experience with what you describe in the last paragraph. The use of textual emoji is indeed introducing another variable, so I’d value an answer even if it used the same images everywhere. – Crissov Oct 23 '16 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.