6

There is a bit of a split on this topic within my team. We use usertesting.com and when testing our prototypes we have opted to not notify the users that they are testing a high-fidelity (flexible, polished UI) prototype hence not every button or interaction is functional.

The UX lead has said that we shouldn't tell users as it will set up a bias and encourage our users not to explore much as they will think that any mis-click or button might not be functional. We can always gain insights as to where the user is trying to click.

I can't disagree with this logic but I have seen other tests and an article stating that it is best to tell the users that they are testing a prototype. I can also see frustration when a user tries continuously to click on something that wasn't 'built' in.

Any best practices that some stick by?

4

If you don't want to immediately make the user aware that they're not using a finished product, then you should definitely make them aware when they go to click on a button (for example) and it isn't yet functional.

I would strongly suggest that you pop up a tool tip or another form of (non-obstructive) message that informs the user that the feature they are trying to access is currently in prototypal development and should be working at a later date.

This stops users from feeling that your application is poorly made, because who on earth would release a final product with buttons that are broken? This also prevents wasting a user's time if they feel the need to report the feature as non-functional, because they will already know it's under development.

Edit: You shouldn't say that the entire application is a prototype - rather, just that the feature they're using is under construction. This will mitigate any sweeping judgement on the application as a whole because it partitions the user's expectations.

  • 1
    Ah.. I really like using the term 'Feature is under construction'. I think this will help make the user feel better towards the brand if something does not work as well. Thank you :) – Usbman Jan 30 '17 at 23:29
  • +1 That's what we do as well. We make all buttons functional, but the ones that lead to functionalities or features that are yet to be implemented are accompanied by a pop-up explaining just that. So far this has really helped tone down the confusion and amount of "Why is it not loading" said during a test. – Wanda Sep 15 '17 at 11:36
3

This comes down to how you conduct your testing.

Personally, I always inform my test users that they will be working with a prototype so they cannot 'break' the system and will not destroy any data. This makes them feel a little privileged, helps to set them at ease and allows them to move with a little more confidence. It also means that they don't end up with me saying "sorry, that doesn't work yet" over and over.

I also carefully script my tasks so that, if the design is successful, the test user should not encounter any non-functional parts/controls. You can consider the user hitting any non-functional part of your prototype as a design fail - either of the product or the test.

I find that, if you hide the fact that they're working with a prototype, users feel that the product is buggy and that in itself becomes a distraction that can colour Kano, SUS and NPS scores.

  • 1
    +1, Exactly the same for me, nothing to add – Devin Jan 30 '17 at 18:46
  • I agree with you, however for now we do the bulk of our testing on Usertesting.com which means we have no control over the script once the user starts the test. It gets super frustrating for both parties watching them click a button over and over again. I love your description though which I might try to communicate with the UX lead here. Thank you. – Usbman Jan 30 '17 at 23:26
  • @Usbman, If you are testing a specific task then your questions and the task description shouldn't allow for anything that might unduly influence the test - You need to pick your wording and direction very carefully. This is true for any task-based testing. If your prototype is built correctly, your design is correct and your test is correctly designed then the user should be fully able to complete the task without hitting any non-functional areas. You can consider any non-functional hits as a failure of either the design or the test. – Andrew Martin Jan 31 '17 at 8:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.