Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I work on a CRM system for healthcare and there's a feature where we send an SMS to a patient reminding her of some scheduled consultation. The patient can then answer the SMS with either "yes" or "no" to confirm she's gonna show up or to cancel the consultation - the system tries to recognise a positive/negative answer by doing some very ingenuous and cheap checks and then updates the status of that consultation.

The problem we have is that users don't always answer "yes" or "no" - despite the fact that that's explicit explained in the body of the SMS; they very commonly answer something like "oh, very nice, thanks, I'll show up" or "gosh, I'm stuck in a traffic jam, won't do it", or even "nice, can you confirm the address please?".

That happens a lot, and we can't identify those as positive or negative - not without investing in some heavy natural language processing, which is not or point. The challenge here seems to be making clear for the user that he should answer with only "yes" or "no", but we're failing on that.

The current SMS text follows:

#{patient_name}, you consultation is scheduled to #{consultation_date} at #{consultation_time}. Answer for free YES to confirm your presence or NO to cancel it. #{doctor_name}

Is there some way we can assure we clearly instruct the patient how to answer while dealing with a 150 characters limitation?

Edit:

Currently, when we can't recognise an answer, we send it as an email to the clinic explaining the situation. That's OK, the clinic can then deal with it manually, but that's kind of killing the automatic confirmation/cancelation of scheduled consultations.

share|improve this question
    
If the system can't understand a patient's response, does the patient get another message to explain and repeat the instructions? –  Matt Obee Jan 31 at 20:52
    
No. But we forward the answer to the clinic through email. –  Fuad Saud Jan 31 at 21:47
    
Is that literally (copy & pasted) the text you're sending? A copy writer can clean that up a lot. –  Dogweather Feb 1 at 18:29
    
You're message reads as if is from a human. Simple prepending the words 'AUTO REMINDER' might go someway to telling people it is a machine they are talking to. –  Brendon Feb 2 at 19:07
    
@Dogweather it's pretty much it, only translated to english. –  Fuad Saud Feb 3 at 0:00
show 1 more comment

7 Answers

Don't make the message sound like it's coming from a person and make the instructions the last thing the user reads:

"#{patient_name}, your consultation with #{doctor_name} is scheduled on #{consultation_date} at #{consultation_time}. Are you still able to make this appointment? YES/NO"

This is better because the last thing a person reads in the message is the instruction for what to do.

The current solution has the doctor's name at the end, which makes the message seem like the doctor personally sent it.

While it creates the impression that the user is getting personalized attention, it is also encouraging personalized/conversational responses.

You would need to test my suggestion above with actual users and see if the response rate improves of course.

share|improve this answer
2  
The doctor name at the end is really something that would make the user think it is more personal. Charles suggestion is great. You could also fit in there, maybe at the end, "This is an automated message." ... to make sure the user understands it is not something personal. –  KuramaYoko Feb 1 at 0:34
2  
I say, make the instructions the first thing. I believe the further you get from the start of the text, the less will be read. –  Dogweather Feb 1 at 18:24
    
@Dogweather is correct - on all phones I've had, long messages get cut off in the middle and you have to "view details" or similar to get the whole message. The message above would end with something like "Are you still able..." and the YES/NO instructions would never be visible because most users wouldn't bother looking at the whole thing. –  Izkata Feb 1 at 18:48
1  
I agree with "don't make the message sound like it's coming from a person" (who will understand free-form replies), but I don't think your example improves it; it just makes it sound like it's coming from a brusque and rude person. –  AmeliaBR Feb 1 at 21:11
add comment

Never send a human to do a machine's job

Ref: The Matrix (1999)

Instead of trying to get users to answer a specific pattern, change the message to a reminder, where users who can't make it gets a one click phone call link in the message. No confusion, no wrong interpretation and if users make the call by mistake, it'll be solved within a minute.

#{patient_name}, you consultation is scheduled to #{consultation_date} at #{consultation_time}. If you need to cancel this reservation, please call us at #{cancel_phone_no}. Welcome! #{doctor_name}

share|improve this answer
add comment

Simple answer: You can't.

Any time we allow open ended responses, there is the risk of getting strings you don't recognize - it's inevitable. Instead of trying to solve it, you should come up with a good solution to mitigate problems that arise from them not entering the correct text.

A follow up message asking them to please respond with the correct input is one way. You could also have messages that don't fit into yes or now be routed to and individual or service (like mechanical turk) that can manually say whether it was a positive answer (yes), negative answer (no), or neither. You could create a much richer database of "yes" and "no" answers off of this to interpret answers in the future. You'd still need a solution for those unknown answers though.

A different way to tackle this would be to understand why people aren't texting in the correct answers - are they older? do they not understand texting? If this was the case, maybe using a service like Twilio to do automated phone calls and having the user input a 1 to confirm would work better.

Just ideas, it's always hard to craft a solution given minimal details.

share|improve this answer
3  
With a phone call, you introduce a new problem: what happens if the recipient cannot take a call at the time? They might be driving, or away from their phone, any number of things. A text message can be responded to at their convenience; a phone call is more insistent. Perhaps the phone call can be a followup for an invalid response, but then that sounds overcomplicated. –  Bob Feb 1 at 18:47
    
I don't know if being insistent is bad in this case. It would be much better for the customer and the doctor's office to have a confirmation. Showing up and your appointment has been cancelled or not showing up and costing the office time/money are both worse outcomes. There are a lot of different solutions possible, the best way to know would be to MV test over a period of time. –  Willie Morris Feb 1 at 20:57
add comment

Perhaps you can switch the message to a one answer system. Message might read "Reminder: appt w/ Dr. Abc this Thursday at 2pm. Reply if you CAN'T make it."

When we switched to that format for dentist reminders, we saw reply volume drop about 30% (because we didn't have to handle vacuous yes), false responses drop 8% (to 1 in 9 being vacuous), no-show percent remain stable, and the overall quality of the interaction improved because most "no" replies requested callbacks.

share|improve this answer
    
Interesting. We don't currently have the habit of monitoring stats about the responses, that's actually pretty good idea. –  Fuad Saud Feb 3 at 4:09
    
Yeah, the responses will point out the kinds of problems people are having so you can adjust your message accordingly. Alternatively, you can send different kinds of messages and observe the responses. Doesn't have to be formal MV testing. –  bishop Feb 3 at 16:18
add comment

I don't think you can fix the way people respond - you are giving a clear direction in limited characters, which is either misunderstood or ignored. You'll have that. So instead, I suggest you handle "yes," "no" and "other."

It sounds like you prefer to stick with yes/no answers, which I understand completely. I wouldn't advertise this new "other" option. Any response you get that isn't a yes/no, respond with a message like "We're sorry, but your response must be YES or NO only. If you have any questions about your appointment, please call ###-###-####. Again, please respond with YES or NO." You could also automatically forward non-yes/no messages on to the appropriate email address.

The followup message should help drive home the YES/NO only policy and gives them a phone number to call. Obviously you prefer they not call the phone number, but only the people who are more likely to need it actually get it.

I wanted to suggest a system where you gave users a link to a web form with simple yes and no options, but you'd be hard pressed to do so without a security risk- you'd have to generate a link with a unique identifier per appointment, and if anyone ever figured out how that identifier was generated, you'd be in trouble.

share|improve this answer
    
Thaks for your answer. –  Fuad Saud Jan 31 at 21:37
    
(pressed enter too soon, sorry) We actually have these links (containing a hash) in the email version of the reminder (which is sent to patients that don't have a phone number). The problem is that we cannot rely on link on the SMS's because a very portion of the patients don't have internet enabled phones or have but don't know how to use. I like the second SMS idea. I'm not sure it's worth it though, because the clinics will have to pay for it, and they may prefer to handle it by hand (I'm not the product manager, so I don't have details about that). But that's interesting anyway. –  Fuad Saud Jan 31 at 21:47
1  
I think this is a good suggestion (assuming the cost of the extra texts is worth the savings in staff time), since it doesn't change the familiar first message. However, I would make it more explicit in the follow-up, that the reason the answer got rejected is because this is an automated system. Something like "I'm sorry, our computer couldn't understand your response. Please reply with 'YES' or 'NO'. If you have any questions about your appointment, call ###-###-####." –  AmeliaBR Feb 1 at 21:08
add comment

I think you're simply not being explicit enough about the fact that this is an automated system and that you only want specific words in the response.

Please confirm: #{patient_name} is scheduled to see Dr #{doctor_name} on #{consultation_date} at #{consultation_time}. Reply for free with the word 'YES' to confirm or 'NO' to cancel.

That is only five characters longer than the original. If date and time are of the form "Fri Feb 18 at 12:30" then you have 15 characters each for the patient's first name and doctor's last name without going over 160. The exact instructions are at the end, and so are fresh in memory when the patient presses reply, but "please confirm" is the first thing read, so it's clear that a response is required.

If that doesn't give a valid response, you can use @Surreal Dreams' suggestion of a follow-up with a phone number in case they do have a complicated message to communicate with staff. The version I gave in comments is a bit too long, this fits:

Sorry, our computer couldn't understand your message. Please answer 'YES' or 'NO'. If you have any questions about your appointment, call ###-###-####.

share|improve this answer
add comment

From a UX point of view, I agree with the other answers given: you cannot rely on customers giving you one of the preplanned responses. You can make it more explicit that they are interacting with a computer, and you can send follow-up messages (possibly at greater expense to the customer, if they are paying for the SMS messages), but some percentage will always ignore what the computer wants.

From a customer experience point of view, if the health care provider using the CRM system to send these messages has a call center, then route the unplanned responses to a call center agent. (This might happen after the initial unplanned response, or after one or more follow-up messages as suggested by others.) The agent can quickly see if the customer/patient is answering in the affirmative or negative in a chatty way, or needs to actually interact with someone. The agent can then either enter the yes/no response into the system on the patient's behalf, or call the patient to deal with their additional needs.

If the health care provider does not have a call center, then the follow-up messages are probably as far as you can go. If, after X number of retries, a planned response is never received, then treat it as if no response was ever given.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.