3

I’m going to evaluate a website to find out how to optimize different processes of interaction. Luckily I’ve got users of the target group to interview face-to-face under laboratory conditions. Unfortunately they’re not native speakers of the website's language, which is not unfamiliar to them. Regardless of a possible barrier of language I’d like to test them because of their particular profession and local availability which in my view is quite essential for gathering useful insights.

Any advice to compensate for such disadvantage? Should I run a language test or self-assessment questionnaire before?

1 Answer 1

1

It's less optimal than running a test with users who are both knowledgeable in the domain and native speakers, but it's still a very good test to run, and the results will be conclusive, as long as the users have good language skills. I've done it, and I had positive results.

The big advantage of this test is that the users will have somewhat higher lostness than the average native speaker target user. Thus this group will be more sensitive in discovering issues, especially information architecture issues. If you have the resources to address the issues found with this group, your application will be usable not only to users starting at average comprenehsibility skill, but to those starting at a low comprehensibility skill. This is the same logic as major magazines writing their articles at 8th grade reading level even when their audience has better education than that.

The major disadvantage of this approach will appear if the information architect speaks the same language as the users' native language. In this case, false friends will likely remain undetected, and you will need a second walktrough, this time with a native speaker (who speaks the technical jargon of the target domain) to verify that the language is not misleading.

Screening the users is not a bad thing if you have enough candidates that you can pick a subset for your interviews. Note that language skill is not a uniform thing; a multilingual might be able to speak in language X about a given domain, but not really in language Y, even though he knows both languages "well". In the screen, ask specifically if the users use the application's language while they are doing tasks from the application's domain. So, if your app is about horticulture, has an interface in English, and you have hobby gardeners, you must ask questions like "do you read horticulture books in English" and "do you discuss gardening with others in English". General statements like "I have finished a B1 course level in English" are less useful.

Another thing for which you can try screening is pluckiness. A shy candidate, or an alpha animal who never shows a weakness, will try to hide his confusion when he misunderstands the language on the screen. A plucky candidate will just ask you "what does this mean" or even tell you off "you should have used an easier word here". But I don't know a good way to screen for it. If you are recruiting through personal contacts, you can ask around in your network specifically for people who "are not afraid to speak their minds" or a similar neutral description.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.