1

I have 2 doubts about usability testing.

1) For example, if we prepare a scenario for the user create a new conference in the system. We should also inform the data that he needs to use while performing the task? For example if it is a scenario for a user to create a new conference in the system, we should also inform in the scenario description the data that the user needs to fill in for each field? For example:

  • Title: New Post
  • Category: Category

  • etc.. for all the fields the user needs to fill

2) Also if we can't find users that are not representative of the users of the system, for example a conference system if we want conference organizers to perform the usability test tasks but we can't find any, if the system is not complex, have a back end that allows to insert, update, remove, etc. We can just recruit participants that are familiar with sites in general?

If it's OK to use this non representative users, maybe it's better have a questionnaire to check if this users that are not representative (they are not conference organizers) can have the minimum level of knowledge to perform the test. But so which questions should we have in the questionnaire to check if this users, that are not representative (they are not conference organizes) are able to perform the test? Ask if they are comfortable interacting with sites in general is enough?

  • 2
    Whether you should supply the information for the data entry tasks depends on your research objectives. Are you skewed more towards making the UI design efficient for those familiar the domain? Or do you want to make it more learnable for those unfamiliar with the domain? What you need for experimental validity depends on your focus. – Luke Smith Dec 24 '17 at 3:59
4

In general, you should recruit users that have different degrees of expertise within the domain. For what I understand this is a pretty generic CRUD system so it doesn't matter if users have specific experience in the particular market niche. As long as they have ever used a CRUD-based system (and most people that used a computer for at least 5 years have used it at some point), you should be OK.

Of course, you can try to refine and/or get a very broad representation of your target with many different degrees of expertise. That will depend on how specific is your system, so when you recruit users, you need to keep that in mind when you do the screening.

Bottom line is: all products and services had to start at some point, so the initial testing may have been done without a known reference or previous user expertise. Furthermore: less experienced users tend to provide really valuable results because they find issues experienced users won't find. So, as a general answer to your question: the level of expertise of users shouldn't be an obstacle when testing common systems for general public. However, it's important that you consider the level of expertise when extracting insights from your testing

  • Thanks for your answer. Relative to the the first question, do you know if we should inform what users should fill in the test in the scenario and measure the task time considering that? – JohnX Dec 23 '17 at 16:47
  • It will depend on the type of experiment. In general, you shouldn't tell users what to do and let them use the system by themselves, so it depends on what you need to measure. This being said, be careful: guided tests may provide entirely different metrics. As for measuring time, again: it will depend. But it won't hurt to do it, so go for it anyways. Finally: are you considering think aloud techniques? If so, the data you gather is way more important than the time it takes to complete the task (and if you can do it, I'd recommend that for usability testing) – Devin Dec 23 '17 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.