2

Let's say your app has a feedback mechanism in which users can leave suggestions and complaints. At what point do you typically take action on users' feedback? Is there a certain number of times a suggestion or complaint has to appear before you take action? Is there a statistical calculation you use?

1

Short answer

Unfortunately there is not a decisive answer. Feedback must be collected and evaluated case by case. Moreover it's not something you can do alone but an interdisciplinary activity.

If you're a single developer and you sell, let's say, a mobile app then you probably just need to set that threshold to a value you can manage ("can I read 1000 feedback per day? No, let's set a threshold to 10 repetitions, in this way I may low that value to 10 and I can handle it".)

Introduction

Let's first imagine you do not have 1,000,000 user base which sends 100,000 feedback per week. In this case the normal flow depends on company's quality procedures and regulation applicable in your domain. Let me introduce a simplification imagining that there are not specific regulations.

Severity. Not every feedback is equal. A bug may have higher priority than, for example, a new feature (because if, for example, your organization tries to follow a zero-bug policy then you can't even schedule and estimate new features before you fixed all the bugs.) Even if you're allowed to postpone bugs to another release, critical or security bugs may have highest priority.

Unless you're a single-man company this first rough prioritization should be made together, at least, with your Quality Office, Product Manager and Project Manager. Also Marketing should be involved because a minor non critical bug with high impact on company image is likely to have higher priority than something more serious.

Note that to determine bugs severity is a fine art, because many variables are involved (a BoD which happens once per year is likely to be less serious than an annoying bug that happens 10 times per day).

Sometimes regulations impose that you take a corrective action for a compliant...immediately (which is in fact as soon as possible).

Now you have an ordered list of bugs and features.

Value. For simplicity let's assume that you will fix all bugs. Which features you want to consider? In which order? Resources are limited then features must be prioritized.

The first and easier estimation is based on the value of the feature. How many users will use it? Will be in the 80% or in the 20%? Note that sometimes an almost useless or marginal feature has high value because it's attractive during demos. Or because other competitors don't have it. Or because it's required in a request for tender.

Also who gives that feedback is important, is it a big or important customer? As you can see we're now involving more and more departments and roles.

Cost. How much will it cost? A new feature requires to be described and analyzed. You may need UX tests. Architecture must be decided, code designed, written, tested and reviewed. You need to collect feedback on-the-way. More tests have to be written. It must be documented (both in user manuals, on-line help and design documents). Older documentation must be possibly updated to reflect these changes. It must be reviewed for legal issues and for internationalization. Technical support people may need training. Resellers may need training or marketing material. End-users may need training (and this may be a cost or a gain.)

Resource Management. You need to assign resources to complete a feature. They might be (or not) available when you need them. You may need to find new resources or to temporarily re-assign them.

Return of Investment. Impact of a new feature is not obvious. Sometimes only few users will see it but it will, for example, decrease the calls to the customer service or it will make technical support easier and faster. You will invest X and you have X*2 in return. Will a new feature convince more users to adopt your application or to update to the newer version? It also depends on your updates pricing policy and if you have a target for the year/semester/quarter.


What if number of feedback is overwhelming? In that case you probably can't ignore bugs but there aren't numbers because in each domain there are different priorities. Thresholds are somehow arbitrary and must be tuned for your specific case. Any decent software will let you filter the list using some criteria, experiment until you find a balanced result and discuss rules and exceptions within company (if, for example, one of your customers is a military organization then you may want to immediately evaluate every feedback even if for normal users you set threshold to 10 repetitions...)

Then?

At what point do you typically take action on users' feedback?

Because of all the above: sometimes immediately, sometimes as soon as you collect enough feedback to outline a proposed feature, sometimes as soon as possible. Sometimes a new feature will rust in your project management tool for years until it will become relevant (maybe never.)

You really need to evaluate case by case together with other figures in your company.

Is there a certain number of times a suggestion or complaint has to appear before you take action? Is there a statistical calculation you use?

Obviously if you have 10 users and 5 send the same feedback then this will increase the impact and the value of the feature. Because of the above, however, this may not trigger any action. There are not magic definitive numbers because cost of each feature is different and value of each user might be different too.

As you can see in the short answer section even if you're a single developer you can't pick a number someone else suggested (I'd not trust any study in this sense) because the number you can manage is not the number an average persona can manage.

0

When something becomes a pattern of behavior likely to impact reviews, which in turn impact app purchases, then it's probably actionable.

Everyone's action threshold is different. Let your priorities make the decision.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.