5

A colleague of mine would like to get some feedback on a new interface that we've already launched. The feedback would be given by 5-10 users, remotely, and unmoderated - we'd give them the interface and then ask them to complete some sort of feedback on their own, in their own environments.

What approach should we take? A questionnaire? A few tasks followed by a satisfaction scale? I've tried to get my colleague to lock down the goals of her testing, but I think it's generally to see if the platform can accomplish what it's supposed to accomplish and whether there are any pain points for users (including look and feel).

3

For remote testing there are a number of options.

You could combine those methods you mentioned: questionnaire, satisfaction survey, etc..

However, the most insightful method for remote testing would be to get the users to make a recording of their screen during the test, preferably with microphone audio. That way you could conduct a sort of remote think aloud test.

| improve this answer | |
3

The classic approach would be to setup some scenarios and record the screen and audio and see the users using your interface while thinking aloud.
The screen and audio recording is a must in order to evaluate the test. A facial video over a webcam is a nice-to-have, but will lessen the possible testuser.

The tool for a unmoderated remote session is more complex than a tool for a moderated remote session such as GoToMeeting. The tool should guide the participant through the session and records what happens, so play the role of a session facilitator.

For myself, I use Userzoom or also used WhatUsersDo.
For more tools and a more in-depth guide what you need to know about such a tool, read the article by nngroups about such tools.

If you only want to make a analytical test, like "do users find the new function", you could also use a tool which only captures the click stream. But for this, you'll need more users to make a valuable quantitative test.


How to run a unmoderated remote test?
First of all, like in every test, you need to fully understand why you want to test the interface. So define your objects of your test. E.g:
Test object
Evaluate the booking process
Test Questions
Do participants understand the field labels?
Do error messages support participants to progress?
In this way, you'll set the focus for this test and you can think about good scenarios.

Next you should think about who you'll testing with. Are the participants real users of the interface?

Then the most important part, the scenarios.
The scenarios should motivate the user to do something. You can't expect a user to start using your interface without e specific goal or task. It's the same when someone gives you a clean sheet of paper and says "draw something" in order to see how good you can draw. It's better to give the paper and say "draw me a horse", so the user knows what to do and when he finished the task.
The same goes for scenarios and task. Specify a clear end of the task, don't make it too complex and make it easy for the user to understand.
At the same time, don't write a instruction. Don't write exactly, what the user should do, since it won't be a test.

To show you better what I mean, I'll state an example:
You have an interface to book flights. Your goal is to test, if the new workflow is working.
Here is a bad example of a scenario / taks:
"You and your husband want to fly from London LON to Tokio TYO. You will fly from 1.1.2018 and return on the 15.1.2018. Then click next. On the next page, opt-out the checkbox for the premium guests, since you aren't one."
With this scenario, you guide the participant too much. Make it open with a clear target. Like the following:
"You and your husband want to fly to Tokio on 1.1.2018 for two weeks. You don't have the premium feature but want to sit side by side.
How do you proceed?"

Write some scenarios and give it to the user. They think out loud what they think while completing your task and you can see, how they behave.


To finish up the whole testing, you can provide a survey about each scenario and the whole test. It's to know how the user felt about the tasks and the interface, and you'll learn a little bit more about how to make it better next time.

I hope those tips will help you get started. There are more ways to test remotely, simply google about it. The way I just described is the way we always do it in my company and we're happy for a couple of years with that. But each project is different, so you'll adapt the course of action.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.