Should we kill the features that users are not using frequently, to improve performance?

Question

According to the heat map that we generated for our app (an ad server), users rarely sort the items in the lists, they usually use the search to find the items they wanted to find. Developers said it will highly improve the performance of the app if we remove the sorting feature.

I am very skeptic about this change as it's a best practice to be able to sort the data in the long lists ( our lists can reach to 100 items) but in the same time the performance (loading time) of the app is very important for the company.

Any suggestions?

Notes :

Thanks to kettch for these questions :

Do your users know that the columns are sortable? - Yes in the user tests almost all users noticed the sortable header

What is the average size of the data set? - Usually, you get 50 lines per page, but a standard campaign manager (our targets) only need 4 - 5 items per page for now

Why are the users who do sort using the feature? - I don't have a clear answer for that, I would say they try to find underperforming campaigns by sorting them by pacing or CTR

Is the data grid so painfully slow that the users never user it? - No it is fast actually, but – apparently – the sorting feature is causing problems in the database and creating discrepancy in the data

Developer's notes for the issue :

• Issue getting the data in postgresql because of the amount of calls needed to update this data, just so that we can sort it, if less than 5% of our users are actually using this to sort.
• Won't scale forever the way it is now

Also what we would be winning by doing this the way we are planning :

• Realtime. Like, really real-time. Every time you refresh the page you would see the latest performance (and not 2 minutes after or whatever)
• Better performance not only for the list, but for the whole product. Less "freeze" in part of the apps
• Cost optimisation, because we wouldn't need to compute it every 2 minutes, although no one would actually see the data (during night or whatever). We would compute it only when necessary

Answer 1

Beside what was said in the other good answers here, you have a much more basic problem. You are misreading your data.

A heatmap generally sums up all clicks on a pixel, regardless of who made them. And you (and the other answers) seem to be interpreting this heatmap as the proportion of users who ever click on that pixel, which is an entirely different cup of tea.

Imagine that you have an online shop. A standard heatmap will display a ton of clicks on features related to browsing around, looking at comments, etc., and only very few clicks on the "Checkout" button. This does not mean that nobody uses that button, it means that people click a ton of other things during a visit, and then only place the order once. And this is exactly how you want them to act! In your case, it is entirely possible that every single one of your users is using the sorting, but only need to do it once per visit - and still view it as extremely helpful for their task. (I am not saying that this must be what is happening - but with your data, you cannot rule it out).

To get the information you want, you would have to generate a completely different information display, which relates clicks to the user who made them, and only counts a click on a given element once per user. This will tell you how many of your users don't use the function.

Once you have gotten that information, you can start considering whether a function used by few visitors should be removed or not. (If you find out that only 10% of the users ever checkout in your webshop, would you remove the "Checkout" button?)

Answer 2

Performance is important, but even more that your goals are achieved.

Consider what kind of users are utilising the sorting feature. Because, for example, it might happen that those users, although few, are the ones that you are interested in supporting.

I would suggest A/B testing to see how does removing the sorting affect your goals. You might find out that you want to boost the search functionality but still keep the sorting.

Answer 3

Do your users know that the columns are sortable?

I ask this, because even though there appears to be a sort indicator on the first column, the users may not realize that they can click the headings.

What is the average size of the data set?

If, after a search, I get all of the information I need in a single screen of data, I might not be inclined to sort.

Why are the users who do sort using the feature?

This is sort kind of related to the previous question. Not all users like to search. I've seen users just take the default data and sort 'n' scroll their way to happiness.

Is the data grid so painfully slow that the users never user it?

Is it possible that the users just don't like sorting because of the performance problems? As a developer who dabbles in UX, I'm not sure I buy performance as a barrier. Unless your application is performing huge aggregations every time they sort, there is no reason why you can't aggregate the data and then sort the result. You could even periodically pre-calculate the aggregations. The very presence of sort should not be an issue.

Answer 4

This may be a bit off topic as it more sits under the development side of things. As a full stack developer I can say that search functionality can be performance intensive. This all depends on what is searched, how much is searched, how much filtering is initially done, etc. I'd have the developers reevaluate the initial search functionality and see where it can be improved. There is generally always room for improvement.

This comment which probably came from the developers isn't accurate if things are being done correctly. "sorting feature causing problems in database and creating discrepancy in the data" This comment alone would make me weary of them. To me this is a cop-out for being lazy and not wanting to improve things.

I wouldn't remove the options if they are important to the page as their isn't much in way of search options to start. I see only 3 filters and this shouldn't be a significant performance hit.

It may also be that the placement of the filters is affecting their usage and changing this around may give different results. You may consider which filters are the most important for the user to obtain their goal and adjust as needed.

Answer 5

You should always be extremely cautious about removing a feature. Most companies don't have a very good idea of why their customers choose their products over their competitors'. There's always the possibility that you will accidentally remove a killer feature and put yourself out of business. You need to have a very good business reason to remove a feature. From your question, it doesn't sound like you actually have any business reason to remove the feature. So the short answer is, don't remove it.

Now if performance really matters, as in your customers and complaining and then leaving for your competitor because your product is too slow... then you can start considering tearing parts of the app out to fix performance. That is an actual business decision with pros and cons and expected outcomes that need to be calculated and considered. In that case you need to consider how much money are you actually losing due to losing customers plus how much will it cost to fix the issues vs how much money / how many customers would you expect to lose if you remove feature X in order to do so.

But this is a business decision that requires a lot (and I mean a lot) of specific information about your business and your customers. Basically, it's unlikely that we can answer that for you.

On the plus side, from your question it sounds like there is NO REASON WHATSOEVER to remove this sort feature.

Also what we would be winning by doing this the way we are planning :

Realtime. Like, really real-time. Every time you refresh the page you would see the latest performance (and not 2 minutes after or whatever)
Better performance not only for the list, but for the whole product.Less "freeze" in part of the apps
Cost optimisation, because we wouldn't need to compute it every 2 minutes, altough no one would actually see the data (during night or whatever). We would compute it only when necessary

None of that has anything to do with, or is related in any way to, sorting. In particular, I'd like to address "and not 2 minutes after or whatever" and "because we wouldn't need to compute it every 2 minutes". This very strongly suggests a major issue with your DB architecture. First off, you should never store computed values in the DB because then data consistency becomes a nightmare. Since you also happened to mention: "apparently, sorting feature causing problems in database and creating discrepancy in the data", I'm going to go out on a limb and say what's really causing your data discrepancy is storing generated/calculated values in the DB.

But it's actually much much worse than that, because it shouldn't even be POSSIBLE for sorting to cause backend data inconsistency. Sorting is done as part of select statements, which are typically read-only. No matter how you sort your search results, the table you got your results from doesn't change.

Answer 6

Some good thoughts have already been shared, so I'll just add one thing I didn't see. While the quantitative data around the features use is important, it doesn't reveal why users are or aren't sorting. It sounds like you as the designer assume some perceived value in the sorting function, so figuring out if those assumptions line up with your users thinking (beyond the numbers) could be an informative next step.

Answer 7

It seems that if you remove the sorting feature users will have difficulties finding under performing campaigns. This sounds like an important action, if not the primary one, therefore, you should support sorting.

In my opinion, the performance gain would not be as great as the difficulties it will create for users. It will be better for them to load 0.1 slower but to be able to quickly find out low/high performing campaigns.

Also, performance could be improved in various ways which do not directly impact the usability. But this is a whole other field.

Answer 8

To me, the key question when deciding to kill off or redesign an existing feature is simple: what will be the net gain in usability? The general rule of thumb is if you don't see a gain of at least 20% (by whatever metric(s) is important for your domain), you probably shouldn't do it. The 20% number isn't important, what is important is that you think about what it means for your app to be usable, measure current user performance against this definition, then measure the effects of your proposed change and decide if the gains justify the change.

It sounds like the change as you've framed it is more to benefit the developers than the user. As a UX professional, I would fight tooth and nail against it. Even if only 5% of users are using the sort, it certainly doesn't make your app less usable for other users. But you can bet you will seriously harm usability for that 5% with direct consequences (angry social media and app reviews, possible loss of customers/revenue, increased support costs, etc...), and for what?

The job of your interface is to solve user problems, not make your developer's job easier.

Answer 9

Before even considering removing a feature (which took resources to make) think about the following:

• Who uses it? Depending on the application sorting might be a feature which only power users use and cutting your power users from a feature is a bad move. In my experience (and I'm pretty sure this applies to many other developers) power users are the ones who give you the most constructive feedback about your product. Power users are long lasting users. They don't just check out your app and then move to the next one. These are the customers you want your user base to be overflowing with. If a user is not satisfied with your app the chances of him becoming a power user are gone through the window.

• Is this an unintentionally hidden feature? Perhaps the currently used design of your app's UI doesn't really show the users that sorting is even possible so most of the users don't really know about it and thus are not using it. Doing something in this direction might yield positive results or at least give you a more concrete evaluation of the usage of this feature. If you application has a tutorial of some sort or at least an overlay with helpful information about what's what, try to add the information about the sorting somewhere (more, if already present) visible. I'm often surprised how many users miss obvious (to me) things. A good user interaction starts with teaching the user in the most fast and seamless way how he/she can use your product. Sometimes not finding a certain feature might actually lead to the user dumping your app and going to your competitors.

As for the development aspects (since I'm a software engineer myself):

Issue getting the data in postgresql because of the amount of call needed to update this data, just so that we can sort of it, if less than 5% of our users are actually using this to sort.

If something is seldom used and yet introduces a regularly occurring overhead then something is wrong with the way your calls work. If 5% of your users use the feature than only in this 5% the overhead should occur.

Won't scale forever the way it is now

Multiple options here: apply sorting on subsets and then merge the results (-> sort of merge sort), create pre-sorted views (if the data doesn't change that often you can offer the user a really fast sorted list since you have already done it on the server), put a cap on the amount of data that can be sorted (for example if you have discovered that after 1000 entries sorting produces too much of an overhead limit sorting to only 1000 entries -> allowing the user to sort only a selected subset is also an option) etc.

Realtime. Like, really real-time. Every time you refresh the page you would see the latest performance (and not 2 minutes after or whatever)

I think we have a different understanding of what real time is. If you really have a real-time view of your data (which the user can view through a web page) this would mean that the page has to be updated constantly with very, very minimal latency. In this case the sorting would be the least of your problems (everyone loves using a mobile app which sucks the available (and expensive!) bandwidth dry in no time...). Again what you are describing here looks more like an implementation issue or general design problem with the architecture (backend here mostly) then something related to the sorting feature.

Better performance not only for the list, but for the whole product.Less "freeze" in part of the apps

See previous quotation above.

Cost optimization, because we wouldn't need to compute it every 2 minutes, although no one would actually see the data (during night or whatever). We would compute it only when necessary

This is something which totally depends on the application and the scenario(s) it handles. Since we know nothing of it, can't give you any feedback.

What strikes me as odd is the fact that you mention that your lists can reach even 100 items. Let me tell you something - 100 items in a list even for an app is not much. I would say that your developers didn't do a proper job if sorting a list of 100 items (I presume that each item doesn't contain megabytes of data but if so, what is such data even doing in a mobile app?!) poses such a big problem. I know nothing of the technology you are using (JS+HTML, Qt, Java etc.) but there are many ready-for-use list components which are optimized for handling huge number of items. Perhaps your developers have picked the wrong tools for the job?

Answer 10

Keep the interface as it is, but ask the developers to move the sorting processing to the client-side.

This will take the strain off the database, and the app will run fast for people who don't sort.

Answer 11

No you should note remove this. Because sorting is the important feature you can use it in future with more refined search criteria..Instead of removing it, make it more detailed probably user not getting the filter criteria which they need.

Answer 12

I don't think it is as simple as you should or shouldn't. Is the performance issue perceived or actual? Are you seeing lengthy page load times for search terms that produce a significant amount of results? Perhaps the number of results returned to the page could be reduced, if there is a specific tripping point where X results increases load & sorting by X ms.

Secondly, are other options available. It's not uncommon for developers to apply an all or nothing fix as they will not consider users as their primary indicator as to whether a page functions as expected.

If a problem exists as indicated in the first paragraph, could there be efficiencies gained in improving the way the data loads or the data is held in the cache? Could the data connection be made more efficient?

You'll notice a pattern thus far in that I've not yet suggested affecting the user, unless there's a specific deficieny in the system.

If there is a problem in load times and the load times cannot be improved in the back-end, could you allow users to disable sorting in their preferences with the benefit of reduced load times and performance as the sweetner, also informing them they can turn sorting back on at any time?

Answer 13

One, sorting is a feature that users will take for granted. So no user will ever tell you how great that sorting feature is - they will tell you and be very unhappy if that feature is gone.

Two, I'd be very, very careful if a developer claims that removing some feature will make an application more efficient. (And I have an application on Apple's app store that has no problems whatsoever sorting about 10,000 items in various ways on an iPhone 4 which is about ten times slower than a modern device, so I'd ask WTF these guys are doing if sorting slows their application down).

Answer 14

Good answers already, but I can't resist adding my two cents.

The problem is that the percentage of users using a feature is not an accurate metric of the importance or value of a feature.

This question isn't new to software development. I think every business faces these kinds of decisions at some level or another. For example, in a previous career I ran a bicycle shop. To be able to support customers with strange bicycles and equipment I would often have to buy tools and equipment that would likely never generate enough income to pay for themselves. So let's say there's an en expensive tool that will probably never pay for itself, the return on investment, at least in the first permutation, is terrible. Should I buy it?

The answer is 'it depends'. In the bike shop, the problem I faced was that if I didn't support a broad enough base of customers they would go somewhere else that was better equipped, and once they had a relationship with another bike shop they would likely go there -- even for the easy, high return on investment stuff. If you can always solve a customers problems, they are likely to come to you first. They are also more likely to recommend you to friends.

When you are deciding the return on investment of a software feature, similar thoughts apply.

• When a potential user is evaluating your tool, they might be comparing feature lists, even if they never use a particular feature they might be comforted by knowing it's there if they will ever need it.
• Let's say a company is evaluating the use of your tool. 99% of their users don't need feature X, but ONE person does, and they can't live without it. They will choose the tool that provides feature X, although the statistic of how many users use X might seem to indicate it's an unimportant feature.

It's easy to come up with other examples of how percentage of users using a feature is an insufficient metric for making such a decision. Instead ask yourself how important is your feature to your users and your business model? That's a harder question and probably requires some perspective into your domain and your users.

Answer 15

My main conclusion from your statement is that the sorting is not intuitive enough for all people to utilise it. Searching is slow and time-consuming, especially when waiting for the results it seems. My suggestion would be not to remove sorting as an option, but to further develop it into a system and style that users are more than happy to engage in, and a clear goal would be to improve that heat map with regards to filter/sorting functions.

Answer 16

There are a lot of good answers.

In case none are adequate how about a different approach:

Rename the current version. In place of the current version put a version that doesn't have the sorting, but it does have an "enable sorting" button. If they click that they go to the old, expensive version, otherwise they use the new, cheap version.

Answer 17

The heatmaps are very limited. I randomly click on stuff all the time, and half the time it's not fruitful. It's better to look at feature usage by user. That is to say: which of our users use the sort? And an even better study is why do they use the sort?

Take sorting on Amazon. It's a dumpster fire. People search, and the search results are all chaff, so they try to sort to separate wheat from chaff. And it never works, because there's so much chaff! And that's something Amazon should look really hard at. Filtering, tiered sorting, whatever.

So the better question is when people sort, what are they really after? And how do we better deliver that?

I completely agree, you should not be doing that database-side. The database engine is a precious resource. Programmers shouldn't throw heavy jobs at the database simply because it's easier to code.

Even better, do the sort on-request right on the user's PC/device. That saves network transaction time -- a classic design blunder is to test on the gigabit ethernet in the office, while your user has an iPad 2G falling back on EDGE cellular with 11% packet loss. So network activity matters greatly to usability. If you phone-home for usage reporting, make it a background action not a critical-path.

Answer 18

I agree that you shouldn't remove the feature. I've seen this before, where some manager removes a feature because he thinks nobody uses it. Users start complaining... so the feature is back, but with some new bugs, becouse coders changing other features has omited these "nobody uses" functions.

Answer 19

The Problem: I have been on a team with this problem, basically the backend performance didn't catch up to the features requested. Management asks for a feature like sorting and someone on the dev team says "sure thats easy to add" and may have not even bench marked real world data. A test server can have a 1000 rows of test data on the backend but the live users could have millions and now you devs might be scared looking at their servers starting to catch on fire when someone clicks that button a handful of times a day to sort.

The solution: First off are you seeing or getting actual performance complaints or are the developers just looking into the future and yelling fire? If the product is currently or is about to descend into chaos then by all means remove the sort and talk to the users who do sort And talk to the developers until a solution can be found. If there is some time as in months or years until the sort causes serious product harm then the best thing to do is have the developers fix it, add a index, find a different type of database that will help. I have been there and we let users wait 30 seconds for a sort for a few years but eventually we cloned our data to a new database type "elasticsearch" that opened our world up to fast and instant search/sorting/counting.