1

I'm looking for some advice on how to handle testing when a scenario involves a user needing to choose a specific location (City) in order to find content to complete a task. Content for the site being tested is geo-gated, forcing the user to choose their location prior to viewing content. Content is very specific to each location.

If I were to provide the location to the user as part of the scenario, the task would be fairly easy to complete. If I do not provide location information as part of the scenario the user will most likely fail the task.

Both options bias the results in one way or another. Tests will be unmoderated, so I won't be able to provide any coaching along the way if a user gets stuck. Is there a good way to word a scenario to avoid skewing results in a case like this?

  • not sure what are you looking to test. If you know one of the options will be fairly easy to complete and when you test it your hypothesis is proven correct, then your presumption was right, therefore the feature you're trying to test is good. Or am I missing something? – Devin Sep 29 '17 at 17:28
  • It's possible I'm overthinking it. The issue is that in a real-world scenario, a person may live in near multiple service cities, and choosing one of those cities results in content not being found, while choosing another provides the content they want. Directly telling them the city to choose eliminates the navigation problem all together. Basically it fails to address the assumption that the geo-gate and siloed content is a bad experience. – Sullivan Sep 29 '17 at 18:18
  • 1
    OK, but in any case, you're testing an interaction. You'll always have some kind of bias, but the one you mention shouldn't have much influence in your test results, because that is exactly what you're trying to measure (if I'm understanding you correctly). In other words: if the interaction works, it works, bias or no bias. However, if you want to compare different interactions, that's a whole different story – Devin Sep 29 '17 at 19:19
  • Every research question has bias baked into it. It can't be avoided. A good experimental design won't eliminate bias, but it can help keep it from creating extra noise unrelated to the phenomena you're studying. It's difficult to understand your dilemma without more detail. Right now it sounds like you're saying approach A will make the task too easy, but approach B will make it too hard. Is that correct? What is the question you're trying to answer with this research? – Luke Smith Oct 1 '17 at 17:42
3

Grab a few people and conduct a pilot. That way you'll get other people's perspective and insights, other than yours.

  • I tried to do that with my project team. They have been exceedingly unhelpful and lean toward the opinion that I'm the UX guy, I need to figure it out. But I definitely appreciate the support. – Sullivan Sep 29 '17 at 18:38
1

Is the purpose to detect whether geo-gated and siloed content is bad? If so, then telling them the location as part of task scenario defeats the purpose. Just tell the participant that they live in "x" city that is not on the list you are providing. Then they will be forced to select one of the nearest cities that are provided, in order to locate the content.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.