2

I'm at the early stage of preparing a usability test with several individuals. One of the pages we want to test is the accreditation process, used by journalists who wish to get permanently accredited at my organization. I know some journalist willing to help out and be interviewed while performing this task, however I'm afraid the results won't be accurate because most of these individuals have already been accredited, which means they have dealt with that process at least once.

So I was wondering whether it's better to test users that are not journalists? or should I stick to journalists, since they will be the primary users of the accreditation process?

thank you in advance!

  • Thank you all for all your kind replies. I wanted to update on one of the questions, how will I report back on the findings? I am using a tool called Airtable which is sort of an interactive Google sheets, following the template from Zapier. Here, I add all of the feedback that I am collecting (including feedback from press officers whenever a journalists can't find something on our website -- I list these issues too so we have a repository of journalist feedback collected informally by our own staff). Here's Zapier's template: airtable.com/tblUWAnFV6yLliEcW/viwWQKeNUhKxfiChO – TNT Feb 19 '18 at 16:17
2

I think that the experienced journalists will probably provide good expert insight, but I would try to mix it with not-so-expert users too.

If you're struggling to find these kinds of people you can select some randoms, people who have no association or expertise in the field. You might be asking "how will this help me?" - well you can use personas, for example, to help random test subjects get into a certain mindset without them having the real experience with the process.

  • thank you very much, Rob! I appreciate your answer. I agree with you -- I will mix experienced users with inexperienced ones. I hope this will help us cover most of the problems that users (in general) face when trying to get accredited! – TNT Feb 15 '18 at 10:18
1

Seems like you don't need to test with only journalists because the goal of the test is to test the accreditation process. You could recruit journalism/communications students, people who deal with writing and content - editors, marketers... and experience with online transactions. It's really difficult to find people who have the time to take part in a usability test. You could use those who don't fit into your user types as participants in the pilots.

Those who are actually journalists esp. those who have gone through accreditation processes can provide valuable verbal insight and feedback.

Before you start collecting data, think about how you will report the results. I think every person who actually turns up for a user test provides useful insights, it's the conclusions that you draw (or not) that matters.

Short answer: Recruit as many people as you can so you'll have a reasonable number of results to work from.

1

Have you defined the user personas already? If so, always try to usability test a representative sample of these personas (primary, secondary, etc).

As you said: stick to journalists, since they will be the primary users BUT also try to find some representative journalists who haven't dealt with the process before. You'll then get a good variety in the data. Recruiting different journalists shouldn't be that hard -- research online communities and offer an honorarium to compensate for their time.

0

What are the research questions you want your usability test to answer? Work backwards from that to figure out what to test, and with whom.

How selectively you need to recruit depends on how much of what you're testing is truly domain-specific.

If you're testing information architecture, copy text and labeling, industry jargon etc then yes you probably need industry-specific representative users.

On the other hand, if you're testing more generic considerations which aren't necessarily unique to the specific context of journalist accreditation, then you can draw from a broader sample.

Again, start with the questions you need answered. Once you've stated the research goals, you may not find it necessary to screen such specific test participants.

0

Just reading the question I am thinking that there are a couple of things that you will want to understand or test separately depending on how much you know about the users and the application, in effect you may want to know the usability when it comes to:

  • journalists compared to other users (understand context for journalists)
  • accredited journalists compared to non-accredited journalists (understand context for accreditation)
  • previous accreditation process compared to current accreditation process (understand context for current process)
  • general usability (general design issues)

So you need to try and untangle the information that you will gather because how you select users will determine the context that you will be able to compare between different types/groups of users while a more homogeneous selection means that you can only test for less specific usability issues.

Hope this gives you something to think about and helps you work out the best strategy for testing.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.