1

This may at first sight seem an overly broad and basic question. But drill down a little.... its really not so simple.

Measuring usability- easy. Lots of different ways to test and measure the usability of a system

Getting qualitative input about user experience- also pretty easy.

Measuring overall user experience however, for something that is not even being actively used yet- is this at all possible?

Certainly for an already in existence product there are methods. Customer satisfaction surveys, interviews where you ask about parts of the process that annoy them, etc....

But for something which has not yet been launched... is there any way to get a solid, scientifically reliable measure of its user experience?

Does it require having a past version to use as a benchmark? Assuming one has this, what method is possible here?

2

1. Decide what success means

Decide on your metrics of success early. This is the product's soul, the guiding principals for everything you do. Kerry Rodden's HEART framework is a great structure for that.

2. Test incrementally

When you're developing something new (most of my time), you have to test pieces. It's not an "overall" view, but it will guide you to a wholistic experience if you're always driving toward your success metrics.

Guerrilla testing is a great tool for this. You can quickly get an idea of how close you are to the mark. When you have a collection of features that make up some sub-workflow, then you can test it a bit more formally.

You're always building toward the final, wholistic test.

3. Release something as early as possible

I'm not talking about a big splash roll out. Find a subset of your audience that doesn't need everything. Build their MVP and get the alpha program rolling. That's where the whole picture starts to come together.

There's no doubt that you missed somethings in your incremental tests. Probably a lot of things. With this small group who's offered to help you out, you can piece together the rest of the puzzle. Then you move to beta and a full release when that's done.

Throughout, you're checking components, workflows, then the whole experience against the guiding principals. Sticking to a singular mission* is the only way to predictably reach a successful overall experience.

* There's always some course correction along the way. Just don't go nuts.

2

You could try PRETOTYPING

pretotyping is a way to test a product idea quickly and inexpensively by creating extremely simplified versions of that product to help validate the premise that “If we build it, they will use it.”

Personally, I find it more a great idea than a valid scientific method (well, it doesn't pretend to be scientific), but we've played a bit with these concepts just for fun and some very valuable (and measurable) insights came out from these sessions, so maybe it's the right choice for you

Additional Reading:

  • 1
    Woah, never heard of pretotyping! – Mark Bubel Aug 27 '16 at 17:38
  • Sounds like design thinking (e.g. its core story about the nepali babies and the incubators), though I've never heard this term before. Its definitely a useful thing to do. Alas the organisation I'm in demands data. – the other one Aug 29 '16 at 11:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.