User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
I've seen a lot of articles online about never asking users what they want, not asking users to justify why they want it or why they made a decision, not asking users what they would hypothetically do in the future, etc.
However, I'm trying to find some published research that supports each of these rules to present to a difficult co-worker and I'm really having issues. Can anyone point me to published research that backs these rules up?
(I turned my comment into an answer)
I don't have any published research for you but the reason we generally don't ask those things in interviews is because interviews tend to be geared towards qualitative research whereas, the questions you mentioned are more quantitative.
Usually, in an interview, you are testing a hypothesis (Like "Can the user find how to purchase items easily?"). You will be asking them questions about how they read the interface, what they are attempting, why the might be attempting it in that way, etc. You'll also be conducting only a few interviews like this.
Asking the candidate what they want is no longer testing a hypothesis - It's canvasing opinion. For that to have any kind of statistical significance you need to get responses from a large percentage of your users. You might interview five people who, by chance, all come from roughly the same socio-economic background, for example, but the majority of your users might come from a different socio-economic background. Therefore the opinions of the interview group would not necessarily reflect the opinions of the majority of you users. (It doesn't have to be about socio-economic background. There are infinite ways your interview group could align differently to the majority of your users).
You need a much larger sample when canvasing for opinions.
It's not that you shouldn't ask what users want during interviews, just that the information you get from it will be worthless.
You might find this research paper useful.
http://www.bcs.org/upload/pdf/ewic_hc07_lppaper13.pdf
It talks about the advantages of doing eye-tracking, and then getting participants to comment on the playback, rather than thinking aloud.
It's not strictly an answer to your question, but it has citations that do. You might find that kind of post-hoc playback technique addresses your concerns of 'prompting'. Your colleague's need for answers to specific issues could be addressed by asking further questions at the end.
Personally, I get people to think aloud, and then do a playback for them and get them to comment on that, and then ask my own set of questions at the end.
