Too many reports because report button is too convenient

Question

I have an app that enables users to exercise questions. The users have an option to report a faulty question. The problem is that the users use the option too frequently. The reviewer has checked the reports and it seems that the users don't read the questions. It is much easier for them to report a question than to read it closely. We can't change the questions. We are looking for a solution which will make the users more self-reliant, and avoid false reporting because of laziness.

Answer 1

If the problem is that "...report button is too convenient" and you can't change this (otherwise check Thomas' great answer) then you already have the solution implemented here on Stack Exchange network:

Ask them to write why they think the question is broken. Zach Lipton suggests to start only with this rule and implement the others only if required, I absolutely agree with him: if your community is disciplined enough then easier is better.

There will always be someone that fill with spaces, junk or try to circumvent your checks, your goal is to prevent this without bothering who is using that feature appropriately. Few validation rules you may want to apply:

• Text should be, for example, at least 16 characters length. "Wrong" is not a valid reason. Don't forget they're not browsing random content but questions someone else prepared, they can't be just spam then a detailed reason should be provided.
• Leading and trailing spaces do not count for the length. "It is wrong     " is not a valid reason.
• Four or more repeated consecutive characters are not allowed (or they do not count for the length). "It is wrongggggg" is not a valid reason, also "It is     wrong" is not valid (a trick that reviewers may not note because HTML will collapse multiple spaces.)
• Trailing punctuation does not count. "It is wrong!!!!!" is not a valid reason.

Do not try to make it easier counting for alphanumeric characters unless you're ready to deal with culture differences and text encoding. After you collected few statistics you may add few more rules (however do it after you analyzed current trend, it's useless to add unneeded complexity):

• If an user wrongly flags more questions than average (+1 stddev) then he will need to double confirm and previous rules will be stricter (for example 32 characters). At first I wrote "...user flags more questions than average..." but Francisco Presencia edited this post changing the sentence to "...user wrongly flags more questions than average...", meaning is different but I think he is right and this addresses also Tim Grant's concerns about legitimate usage.
• After a decent amount of rejected flags an user will be temporary banned from flagging.

According to your application's nature and content you may want (as suggested often for this kind of things) to add a gaming/competitive nature using score. Each declined flag will subtract two points, each accepted flag will add one point. When you reach zero you can't flag for one month.

Make it personal, when they submit a feedback remind them of their flagging history ("You already submitted 78 flags and 70 has been declined as wrong".) You may even have different wording when this ratio is above 50%. It's not exactly the same effect outlined in Effects of eye images on everyday cooperative behavior but it surely helps. I remember a nice case study (which I can't find in this moment) written by an high school teacher, he (obvisously some years ago) started to use his notebook in class to keep track of students' progress. He observed a drammatical improvment in their performance (and a drop in their excuses) even if, in practice, to use a computer or pen and paper isn't so different. To know there is a detailed report of your actions is a big deterrent.

Note: keep track of rejected flags and attempts to flag a question, they may indicate that a question is formally right but maybe poorly worded.

Answer 2

There are certainly options which make use of UI design and not only input validation. Don't get me wrong: input validation is also a good point, see the other answers. I just think it's not the only way.

Consider this as the starting point:

The buttons are sized equally, providing the impression that they can be used equally often.

As a first improvement, make it clear that these options are not equal:

Next, add some distance from the submit button:

Also, changing the text from "Report faulty question" to "Provide feedback" could make it less obvious that it is possible to bypass the question:

Answer 3

The fact that your users submit pointless feedback is good. The far harder problem in collecting feedback is getting them to do it at all.

Please improve the UX of your flag reviewers, not your flag submitters. Do they have tools to skip short feedback? Redundant feedback? Is the feedback grouped by category using even procedural language processing techniques?

Never get rid of user data just because they didn't format it well enough for you!

Answer 4

I find it frustrating as a user when it's too hard to flag or report come content, so I think it's good to have it be convenient. On the other hand, as you point, it could increase the number of reports dramatically and that's not necessarily a good thing.

I would extend Adriano Repetti's solution like this:

1. Have a mandatory "reason of report" dropdown and an optional comment text field - you don't need to say it's optional; just don't enforce any particular rule there.
2. Internally, have a set of rules (minimum reason length, gibberish text filters etc) that decide if this report is something that should be looked into (a "good report") or not (let's cal it a "poor report").
3. Good reports can have a weight of 1.0 and poor reports can have a weight of 0.25 so that 4 poor reports by different users are needed to produce any sort of action that usually is triggered by the report function of your app.
4. Some users will report more than others. If a user is an outlier (reports way more than usual), lower their report weight to 0.1 and possibly show them a captcha.

This way you don't annoy most users, keeping it simple for them, and also reduce the amount of work done on your part.

Answer 5

Hello User XYZ,

We are aware that these questions can be quite difficult and involved; they were designed to be this way.

Before reporting this question please consider going through the checklist below:

• Read the question at least 3 times; taking a 30 second break in between readings.
• Move on to the next question and come back to this one later.
• Additional suggestions...

Thank you

If you still wish to report this question then please click the link/button below

Report this question.

Upon clicking "Report this question" provide another pop-up with questions and dropdowns which requires them to validate their decision to report the question.