I just learnt the benefits of standardized usability questionnaires, but the well-known questionnaires such as QUIS and USE evaluates way more variables than I want to know. Let's say I just want to know how satisfied users are with the visual aspect of an app, can I use QUIS and remove all questions that aren't relevant to visual design?


I wouldn't do it.

Designing a question is hard work. Psychometry is a discipline in its own right, and requires lots of knowledge of human behavior and statistic beyond the topic of the questionnaire. There are many things which matter in a questionnaire of which you may not be aware at all, for example the order of questions.

For academic purposes, you cannot claim that you are using a known questionnaire if you modify it. If you use a modified questionnaire, your results are not considered comparable to ones found with the standard questionnaire. Standardized questionnaires have usually been subjected to extensive empirical validation. This validation is not considered to cover any modifications. Even word-for-word translations to a different language are problematic (and may need a cross cultural validation before being used).

Even assuming that you are not going to publish your results, academia has a good reason to be so strict. Using a questionnaire is like baking from a recipe. If you don't know the chemistry of a recipe, you don't know if leaving out an ingredient will give you practically the same cake but without a distracting (to you) flavor, will make your cake's texture feel like bread, or will stop the cake from rising, leaving you with a moist brick. Actually, questionnaires are more insidious than cake. If you fail baking a cake, you'll know it. If your modified questionnaire is not measuring what you think it is measuring, you will still have your data, without knowing that they are wrong. A modified questionnaire might work, or it might fail silently. This is a good reason to not do it.

It is however admissible to apply the whole questionnaire and then only evaluate the part you are interested in, not considering the rest of the data if you don't need it. If you don't have a questionnaire tailored to your needs and use a more general one, you are placing a higher cognitive load on your users, and may have to give them better compensation, but this is what you have to do to get good quality data.

Finally, if you completely cannot find a questionnaire completely suited to asking about one thing you are very interested in, and you are trying to maximize participation by offering friction-free answering (a "please give us feedback" situation like on MSDN articles) you might consider asking a single question which covers exactly what you want to know. The information which you will get will be a) low resolution (you can't get details that way) and b) not as good quality as that from a validated questionnaire. But it might be the right solution for your problem. Just remember to use a single question and to create a very exact, unambiguous formulation in terms the user thinks in.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.