0

Exec. Summary (or tldr): How do I phrase my questions to get the best data?

(This is a follow up to this question.)

I have a very long, paged/saveable form, used by banking agents to onboard businesses. The agents are proficient in a 90s version of this form already - we are upgrading the application to the 21st century.

The form has phone number fields as well as date fields. Currently, the fields are componentized: [MM] [DD] [YYYY] or [XXX] [XXX] [XXXX] Ext[XXXX]. But I'm not sure that's very efficient. It may reduce error but I suspect that it does so at great cost to speed. Remember, these are trained users. For example, it might be better to have the option to copy/paste whole strings of phone numbers, say, from a document they already have.

I think a full-blown usability study might be less useful than simply asking the users. So I'm writing up some questions in anticipation of asking them.

My question to y'all is this:

How do I phrase the question(s) to get the most informative responses?

Should I lead them toward opportunities they might not know are available, or should I not bias them, to get answers that aren't just re-enforcing my own expectations? Both have problems. Examples:

  • "Considering both speed and accuracy, what way(s) for entering dates/phone numbers are/would be most efficient for you?" (Open-ended and unbiased but also not helping them explore possibilities)

  • "Would the ability to copy/paste whole number strings at a time into a single field be useful?" (Helpful but leading them down a path)

  • "Do separated fields provide the accuracy you need without sacrificing speed?" (neutral)

  • "How do you enter these fields now?" or "How do you expect to be able to enter these fields?" (Not useful, since they don't have options or expectations in the current app.)

Of course I also don't want to inundate them with questions, so I have to be judicious with them.

How would you handle this?

1

The information you need to solve this problem is not if copy-pasting is faster and more accurate than typing: very likely, it is and it doesn't depend on user preferences.

The information you need is what proportion of data entry the users do based on digital documents they can copy-paste from, vs analog documents / phone calls, etc. They might have an image scan of the client ID with no OCR: they won't be able to copy a date from it.

These are questions you can ask in your survey: (to be adapted to the specificity of the user role)

  • In the last 24h, how many times did you enter data using this form?
  • Please select all the sources that you used in the last 24h to collect the data: [] paper ID [] oral answers to my question [] image scan [] online form [] pdf or word doc [] other

For each option selected, ask the follow-up question: - in the last 24h, how many times did you enter data based on [option]?

Then you can calculate the proportion across all users.

Why ask the number of times in the last 24h rather than directly asking for the proportion?

Because humans are notoriously bad at estimates, and even worse at making a calculation of proportion. Working memory is very short term, so longer than 24h will give you unreliable answers. The last 24h might not be representative for any particular individuals, but the differences will average out across a large number of respondents.

  • Thank you; I like this answer. I should not be asking the users to guess what they want, nor should I be watching what they do with an antiquated app. I should find out what they have available in terms of resources (eg. e-docs) that they can leverage. – DaveC426913 Jun 14 '18 at 15:35
0

The way you have the questions phrased I would not ask any of them directly to people. You have phrased them more like research questions which you could answer yourself after making some careful observations of what the people actually do in real-life or experimental scenarios.

If you ask people hypothetical questions you’ll get non-data.

  • The problem is that 'what they actually do in real-life' is constrained by the limits of the current application. So my data will say "zero users copy and paste". – DaveC426913 Jun 14 '18 at 15:30
0

Asking your users questions about how they would hypothetically use the new form design is a bit tricky. It's a bit difficult for them to give an accurate answer, because they have to picture the interface in their heads.

You could have them to using various prototypes of the form inputs and get their feedback as they work through it, or listen in on agent trainer sessions to find out how they're training the agents to use these forms.

To find out how users are making use of the forms, you could use a screen recorder and observe them remotely or next to them if possible.

But if you absolutely had to only do a survey, I would re-phrase ""Considering both speed and accuracy, what way(s) for entering dates/phone numbers are/would be most efficient for you?" " as " 1. How do you usually enter phone numbers? [Type phone number into form] [Copy and paste from existing document] [Other - open field]

2.1 Which way of entering dates do you prefer" [show single field] [show split fields]

2.2 Why do you choose this? [This is a more accurate method] [This is a faster method] [Other - open field]

By leaving out the value judgments in the question itself (we assume users want both speed and accuracy) and splitting up the answers into results that you can easily analyze (plus easier for participants to answer)

  • Problem is: the question 'How do you usually enter phone numbers?' is useless, since they don't currently have any alternate options. That's why I have to ask them if it would be helpful - i.e. if the opportunity comes up in their current work to copy/paste (even though they can't take advantage of it). – DaveC426913 Jun 11 '18 at 19:12
  • I think copy/paste is sufficiently well-understood that I don't need to watch them to see if they figure it out in this app. Because these users enter customer data all day long, they will immediately know if the ability to copy/paste is useful to them. A questionnaire would be more efficient than a usability testing session. – DaveC426913 Jun 11 '18 at 19:14
  • You may think that, but it may be such a learned process by now, that experienced users, being presented with a single field, would still not consider using cut-and-paste because they never have before and will fall into their normal habits. – wootcat Jun 12 '18 at 13:59
  • Do the users receive their documents in digital format or hard copy? Or a mix of both? Copying phone numbers from digital documents may still require editing e.g. deleting dashes, brackets etc. unless it's a standardized form (I'm guessing here) This could be a survey question - how they receive their data. – Ling Jun 12 '18 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.