5

My team and I are currently making a program which use camera in the google glass to help blind people 'see' the interlocutor's emotion (happy, sad, angry, disgusted and so on). I need help in case you guys have an experience and know how to help blind people to get the the emotion information. The detection method is working well for normal people because they could see the result (text) in the monitor and now we want to help the blind people to get the same experience. At first, we think that if we might use voice e.g. the person on the left is happy, the person on the right is sad. But then, having voice output would be inconvenient when they are talking with other people.

Thanks for your help

|improve this question|||||
  • 1
    You could try just using sounds that people can relate to emotions such as the sad trombone sound "wom-wommmp". – DasBeasto Jan 11 '16 at 12:54
  • 2
    I'd spend some time with blind people, learning how they understand emotions currently. Don't they do that by listening to the things the person says and their tone of voice? Or are you talking about text interactions? Or just seeing the other's emotions on their face? – Ken Mohnkern Jan 11 '16 at 15:18
  • we will use the camera of Glass to see the interlocutor faces and detect his/her facial expression in order to get the result and report back to the user(blind people). We want to ask if you can help us in order to have best way to help them to have the similar experience with normal people. thanks – akurniawan Jan 12 '16 at 8:24
  • Do blind people have trouble understanding others' emotions? If so, what is missing from the interaction? I'm asking because understanding the problem in depth usually indicates what the solution should be. (I'm also asking out of curiosity.) – Ken Mohnkern Feb 11 '16 at 12:34
1

Looks like there is transformation issue , in this case you want to transform emotional data to blind. I have little bit modified the your voice output in my idea below.

If there is smile transform speech voice output " smiled" (you don't want this)

If there is smile transform into short melody(sound). Like facebook does when you get notification. This will radically decrease disturbance level but you need make run the melodies in order if there is two person with different emotion. In this case you need set order based on which emotion recognized first FCFS policy.

You can assign different melodies to your project.

Happy - Melody1

Sad - Melody2

Angry - Melody3

On the other hand , please have some research on Finger Braille Systems.

Because there is case that a person can be blind and deaf. So my melody idea will be useless :((.

|improve this answer|||||
0

One of the "default" methods for blind people to interact with devices is to feel some kind of indication (e.g. vibrating traffic light buttons). Best solution would be to be able to feel the emotion. I dont know if Glass is able to output vibration but you could try to bind the slider position to the location and output audio according to the visible emotion. Map the finger position on the slider to the location in the FOV the user wants to check, then output a short sound on touch. The user is already able to determine where his conversation partner is, he just wants to get extra information.

|improve this answer|||||
0

If you don't want to add an additional device, the only approach I can think of is related to sound, since this is probably the most developed sense of blind people.

Instead of using words that may deviate their concentration (and being ackward for the other person if the sound is loud enough) you could use short sounds / notes, that can relate with an emotion. (happy could be a high piano note, sad a high or mid violin note, angry a low piano note).

It is the same concept of this video but reduced to a note or a very short sequence of them.

|improve this answer|||||
0

I was just thinking, 'sound' could be an obvious triggering factor. Short sound bites. What about light? how do blind people perceive light ? I am not an expert, but I found this http://www.scientificamerican.com/article/how-blind-people-detect-light/

May be you could use the 'intensity' level of light some how (? Using glass or an added hardware to Glass ?) to train blind people ... too flashy, too bright to indicate 'anger' emotion, Or two flashes (number of times?) to indicate something else. You probably have to limit the number to 2 or 3 times as we cant have users 'counting' in the middle of conversations just to understand the interlocutors emotions. Like I said, just thinking out loud, I am not an expert. Can light be used to indicate such emotions ?

|improve this answer|||||
  • The article you linked to talks about light triggering neurological responses that can be detected by an external scanner, even though the recipient is not consciously aware of whether it's present. This doesn't sound useful for immediate, rapidly changing feedback that the user is supposed to perceive directly. – octern Jan 12 '16 at 12:58
0

N.b. I have no experience of this whatsoever, I just saw your question and thought it was quite interesting.

How about, vibrations that communicate to the blind user a little like Morse code? So little sequences of vibrations that tell the user the emotion that the glasses have detected.

Obviously, the user would need to either learn the sequences or already know Morse code. Also, what are the downsides to constantly sending vibrations to them i.e. annoyance, irritation, pain... ?

|improve this answer|||||
  • Thanks for help, unfortunately, Glass can't vibrate so the idea of Morse code(being converted to vibration can't work in the Glass at the moment). – akurniawan Jan 12 '16 at 9:19
0

Maybe you can use a reduced version of blind-deaf communication system, which is (to my knowledge) based on touch of hands and fingers. Since you have 5 basic emotions and 5 fingers, you could use a glove that receives the input from glasses and transform it in a tactile information (vibration, soft pressure or whatever) to each finger. After some training it could be an easy way to perceieve others' emotions. He's happy=index finger He's sad=thumb and so on Moreover, if you software can detect the intensity, you can modulate this information too on the glove (happy 2= soft pressure, happy 10=lot of pressure)

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.