How do you keep research participants in a small pool anonymous?

Question

For my project, I try to conduct anonymous user research...but it's with folks whose roles as users are so unique, that they can be easily identified simply from their role/demographic information.

To give a specific example, I do research with all people of a particular job type inside of various branches of our company. There is only one person with that job per branch, and we only test sporadically with each branch. Thus, when I give my team the findings from our research at a branch, it's easy to pinpoint exactly who provided what feedback, even if I don't say their name.

Does anyone know how this should be handled? I want to keep being transparent about the research process with my team in order to maintain trust between us, but if I tell them the results, I can no longer maintain participant anonymity, which I rely on to get more honest feedback from them.

Answer 1

If the information can personally identify someone, don't include it in your research. Business role, location, and other demographic info is rarely necessary for internal user research anyways.

If your team is requesting research that specific, then it's on them to maintain professionalism with your users.

Answer 2

Thus, when I give my team the findings from our research at a branch, it's easy to pinpoint exactly who provided what feedback, even if I don't say their name.

This reminds me of those manager performance appraisal forms that say "be honest but not so honest that you can be identified by your remarks."

It is a catch-22. Context like role and demographic info is necessary to qualify feedback as representative and useful and help make the research valid, but to your point it can also make it easier to identify the participants.

So what's really at stake? Is it mostly that you feel you're less likely to get honest feedback from participants who feel they can easily be identified?

Here are a few suggestions to mitigate that:

• Don't ask participants for their names or roles. You probably already have this info, so instead just number the materials and feedback forms for each participant and keep that index handy.
• Don't report on the role & demographics of each individual participant. Instead, just indicate that your participants all met the relevant role and demographic criteria for the study topic.
• Don't report content that can identify individuals. DarrylGodden alludes to this. Instead, try to transform qualitative data into quantitative data i.e. "5 of 7 participants had negative things to say about X, Y or Z..." or even paraphrase.
• Don't leave participants guessing as to how their remarks will be represented. Instead, assure them by showing how the content of your report will be structured with the emphasis on what was said vs. who said it.

I think participants' identity is mostly useful for answering tho question:

"Are these the right people for the research question?"

If stakeholders trust your research planning, the answer should be yes or no vs a list of people.

Answer 3

We number our user testing candidates and the questionnaires they fill in, also when presenting the results you will, inevitably, transform qualitative into quantitative and anonymise the results further.

You can also re-write the highlights so they are not individually identifiable. Be creative with reporting helps here.

Answer 4

You can apply the same way as in witness protection - you simply remove data that can expose one's identity.

Or, as an alternative, you mix the information, the answers become untraceable, but the data remains valid. So the list of participants remains as is i.e. Web developers (2), IT Consultant (2), Server managers (1), but the answers however are not attached to an individual, but a group instead.