Best way to ask a question about a mostly subconscious behavior?

Question

I need to know how much web shoppers in certain countries pay attention to and care about ccTLDs (country code top-level domain e.g. .co.uk or .fr) That is, does it matter to a UK web shopper if a website is called 'website.com' or 'website.co.uk', how much does it matter, and how does this vary by country?

There's hardly any information about this online, in fact, when I search for leads, I repeatedly come across the two questions I've asked on the topic, one here on UXSE, and and another on the Moz Q&A Forum. Other articles on the topic, like this one suggest that "If you use a ccTLD website you can expect higher CTR, higher conversion rates and higher spend per order", but this it typical in that it puts this behavior into the "safe assumption" category.

So I'm simply going to pay for a one question survey and ask 1000 demographically targeted individuals in the countries I care about.

I think if I ask someone from the UK "Is it important to that the website you're shopping on is a .co.uk domain?", they're more likely to say yes because they actually consider the answer, and hence are more likely to rationalize their decision in a way they may not when browsing or shopping online.

Are there any recognized techniques, and/or can anyone suggest ideas about how to ask this kind of question to get the most genuine answer possible? How can I help them channel an emotional answer, rather than the logical/rational one they are more likely to provide?

Edit: I expected answers of the "I've got nothing on this, but let me solve your problem in another way" variety, and that's what I've got. I appreciate that ideally I should observe behavior and am now considering options on that front, but this essentially a question about asking difficult questions. Any input on that topic would be greatly appreciated.

Answer 1

If at all possible I'd try and observe actual behaviour, rather than ask a question. Because people suck at predicting their own behaviour.

For example run a remote usability test with mocked up a google search result around your topic with a mix of .co.uk & .com (or whatever) company names/URLs and see which is clicked most.

You also have to remember that other people's results with country domains vs generic domains is not just based on the domain.

For example, having a multiple country sites means you can make default decisions about language, currency, address formats, etc. on each site that can significantly reduce the complexity of the purchase — so the domain of the site isn't the only factor in play.

Another example, in doing some usability testing for a LargeFashionBrand(tm) we saw a significant minority of users visiting the site of the "home" country of the brand when doing research since they perceived the information there being more current and accurate than the various country sites. They then made purchases either on the country site or Amazon. So just looking at CTRs you'd have seen more purchases on the country site — but that purchase was driven by actions the user made on the home-country site.

This kind of issues make asking questions about domains really problematical — since the domain isn't the issues that's causing the behaviour change. So look to observe behaviour if at all possible.

Answer 2

The problem with this kind of issue is that I suspect you would get different answers from different parts of the globe. I suspect (and this is just a suspicion - I have no evidence for this) that populations with US trade interests may favor the .com suffix as it is considered a larger 'global' entity rather than just confined to their country whereas non-NATO aligned countries may see the .com suffix as an American symbol and would therefore avoid it in favor of their own country's suffix. There could also be more cultural issues regarding the sense of nationalism experienced by any given population at any given time and, with the latest revisions to domain names, linguistic and character-set issues may also temporarily or permanently skew the results of any test.

There are a whole raft of conditions that may affect the result of any testing for multi-national behaviors and I'm not sure that there is a) a definitive answer for all suffixes or b) an answer that will stand for longer than a few years at most.

Answer 3

If it's possible to ask context-based questions, ask this:

Thanks for participating in our survey. Please pick which of the two sites you'd like to help us evaluate:

[www.website.com] [www.website.co.uk]

And adjust the second link to be whatever the local domain extension is.

Ideally you'd randomize the position of both links too, because some will pick the first one just because it's the first.

Answer 4

Have a look at 'Study: How Searchers Perceive Country Code Top-Level Domains' on Moz.com. Author Eli Schwartz presents some successful approaches to really get at 1. whether users are aware of general TLDs, 2. do users see a particular TLD (.edu specifically) as more trustworthy, 3. can users identify a particular location by the ccTLD used, and 4. can users correctly assign a ccTLD to a geographic location. The questions used in this article are great examples of how to approach the question you're attempting to answer. Directly related to your question is this quote from the article:

Digging into the idea of trust and TLD a bit further, we asked the same reliability question about results on Google.com vs Google.de. In the US, 56% of respondents said that the results on Google.de are equally reliable to those on Google.com, and in Australia, 51% said the same thing when compared to Google.com.au. In the marketer survey, 66% of respondents said the results were equally reliable. The fact that the majority of respondents stated that results are equally reliable should mean that users are more focused on the brand portion of a domain rather than its country extension.

Answer 5

PixelSnader suggests the right approach: be sneaky.

For example, one way might be to create a survey on "URLs that create trust," Ostensibly to measure the effectiveness of long vs short, neutral vs descriptive URLs, but to randomly append a predetermined set of strings like

/updates/technology/technews.html
/technews/
/updates/tech/

to either a standard or ccTLD URL and see if a large sample group of users rate the non-ccTLD addresses as more or less trustworthy.

Researchers in the behavioral sciences have long known that the way to measure unconscious behaviors and avoid self-reporting bias is to...well, lie. Or rather, to initially mislead subjects as to what you're measuring and only tell them later, after you've collected the results. A great example of this is, the Invisible Gorilla Test in which participants were asked to count the number of passes in a filmed basketball game. Afterwards, the test moderator asked them not the expected question, "how many passes did you count?" but "did you see the gorilla?" More than half answered "No."

It turned out that midway through the game, a person in a gorilla suit had ambled up to the camera, looked directly at it and waved for a full second. A majority of participants were so busy counting passes that they didn't notice.