Quantifying discoverability on where or how information is found, presented

Question

How do you objectively quantify a discoverability problem if the designer insists it is your fault that you can't find something?

I am an user of this network of sites, which has a subsite for every occasion. They are quite touchy if you ask your question in the wrong site.

The problem is you can't easily tell what site is for what. For example, there is a site for when you are stuck while programming and there is a another site for when you are a programmer. I vaguely remember that in both sites they don't like questions about a programmer's career, but I was not sure so I wanted to double check, but I could not find the subpage that explains what topics are welcome and what are taboo even though I knew it was somewhere there.

For your consideration this is their landing page and this is where you ask new question. As you can see there are links to

• how to ask
• ask on meta
• help centre
• asking help
• formatting help
• and once again asking help

plus header help dropdown and huge footer.

I have scanned some of the links but could not find any eye popping "on topic/off topic" list. So as a good netizen I went to ask on their meta only to discover this is very very common problem. To my surprise, when I pointed our that there may be a discoverability problem I was told that the on topic/off topic info is clearly there under "that link" if only you click on "that menu" (kind of).

How do I quantify a discoverability issue to someone who thinks there is no such problem (despite at least 277 out of 5,658 meta questions being of this sort)?

Is this even a discoverability issue? If not, how would you categorise it? Information architecture?

Answer 1

SE is somewhat unique in that content must be qaulified to a particular sub-domain by some custom relevance. However, we can look at other content authoring sites such as youtube that use tagging in order to decide what searches and browsing your video should show up in, but its optional and doesn't even need to be accurate.

Here the goal should be to either guide the question to be within a particular format, redirect the question to a more appropriate subdomain, or dissuade the question from being asked at all, before the question is asked.

We know the help center isn't very effective, many questions are asked in complete ignorance of the help center. The ask a question page has a mechanism that informs of the accepted topics, but only while the cursor is focused in the topic textbox. We know that isn't very effective either.

What might be worth trying is to preempt the ability to ask a question with qualifying the question within the scope of the subdomain.

1) Tag your question with at least one of the acceptable scopes for that subdomain.

2) (Optional) Justification for a tag if the qualification is iffy.

3) (Optional) Flag your question as requesting a sanity check before publishing. This would put it in a bucket that only users with elevated trust could review/edit before giving you a thumbs up to go ahead and publish.

4) Publish question.

So that every time a user asks a question, they are reminded what the scope of the site is, and ask themselves how their question may fit or not fit that scope.

For programmers.se' particular case, they get a lot of career advice questions. One of the flags available for when qualifying a question could be career-advice. If the user selects this flag when qualifying their question for programmers.se, thats when you can stop the question from being asked, and display a message informing the user that the site is not intended for career advice, and giving directions for where to ask for career advice.

Whatever the solution, you'd want to maintain a streamlined question asking workflow, while lessening the exhausting amount of moderation required.

So in a way, flip the education on its head. Every time a question is asked, a short quiz is taken, with an optional peer review, and the scope of the site is reinforced for that user.

Answer 2

The standard way to quantify these sorts of things is via user testing, using a task that goes along these lines:

You are a new user to this site and would like to ask a question about [some realistic question]. However, you are unsure whether this is the right site to ask such question. Please find the relevant part of the site that explains which question should or should not be asked on this site.

Then, you collect:

• Task completion ratio
• Error rate (mis navigation / branching of the ideal path)
• Task completion times

All these can support a usability issue if such exist.

Answer 3

Its a taboo to critique SE on SE. They've got some questionable security practices that are overlooked/forgiven because we are SE, but we're reluctant to admit they're real, even though we are very critical of other websites that do something similarly backwards. (Because you should know that stackoverflow, serverfault, superuser, etc, are legit SE sites, so give us you SE credentials, you fool~)

And of course you point out one of the biggest reasons for locked questions, the UI. No reasonable, non-biased user will defend the argument "you should have read the help center", we are UX, we know better than to assert that. People don't read the rules, EULA, ToS, or help center. So I think you asked this in the right place, we UX people are supposed to embrace the tendencies of the user, not condemn them.

However its still a taboo to critique SE on SE. I watched a YT video yesterday, the "Taboos of Science", and the Semmelweis reflex topic actually made me think of SE. Our expertise can sometimes get the better of us. How dare you~