4

There are many different organizations starting to collect usage statistics in an attempt to improve the usability or user experience of their software. One of the common frustration when designing an application for a client or organization is that they have no idea of the typical screen resolution of their users, and therefore in most instances you have to resort to a 'fluid/response' design that doesn't necessarily cater very well for any screen sizes. Most of the statistics for display resolution comes from browser data, which doesn't necessarily reflect the organizations that use your application. Therefore, I think it is quite important for software to collect usage statistics to help improve all aspects of the design.

However, when a screen prompt asks you to consent to anonymous user data collection, I think many people would probably not participate even though collectively it may be in the best interest to do so. Are there any ideas or examples where anonymous user statistics reporting rate can be improved, whether it is through the design of the interface or a better wording?

The Steam/Valve Hardware and Software survey in one of the answers provided is a very good example of making the user data work hard to improve the service. Interesting enough I haven't many other examples around, so if you have seen something along these lines please also feel free to share it.

2
  • The problematic part is not only privacy, but also the unknown systematic bias in use patterns between responders and non-responders. Thus wide surveys "on the cheap" are often worse than controlled studies. Jul 15, 2013 at 14:57
  • Agreed, so how can we make it better? Or should we be using it at all?
    – Michael Lai
    Jul 15, 2013 at 22:24

3 Answers 3

3

I think that this question is multi-faceted, and that there is no single answer to it. On a basic level, I think it's important to remember that the easiest and safest course of action for the user is to say "no". To get them to say "yes" (and I strongly believe that giving such information needs to be opt-in and not opt-out), there are many things that must be addressed.

In my experience, it is true that many people don't participate in the collection of anonymous usage data. I have worked on different products with different user bases. The levels of participation in each of those are proprietary information and thus I can't share them, but it's fair to say that many people don't participate. There are many different and valid reasons to choose not to participate:

  1. I don't know what data is being gathered and what is being done with it.

  2. I do not understand what the personal benefit is to me to allow this information to be collected.

  3. I am concerned whether I will experience any kind of negative effect as a result of participating in such a program, such as reduced download speeds when the program uploads my data.

  4. If I'm using a corporate device or network, my employer might not allow (or might actively block) such data from being shared. Alternately, I might not be sure whether my employer allows it and I don't see a very good reason for me to research an answer to this question.

  5. I have to decide whether I think that the company/project/whatever that is gathering the information is trustworthy.

Problems 1-3 are the easiest to address, and are the ones that are most amenable to being addressed in an individual application. The fourth problem is somewhat more difficult to address, but is probably addressable with some hard work and creativity. The fifth problem is likely the most difficult to address, and is probably outside the scope of the work that a designer does on an individual application.

If you are going to deploy an application that collects anonymous usage data, then you need to understand which of these items are the ones that are the most impactful for your users. For example, if one were working on enterprise software, companies that want to ensure that none of their proprietary information is shared inadvertently. One method to address this concern might be to work with an individual customer to gather this data for a set period of time, and to give them the opportunity to check the data that is gathered before it is shared with you. Repeating this process with multiple customers will give you a breadth of data.

If you already have an application that collects anonymous usage data and you feel that you can improve the rate of participation, then you need to determine what impacts your low numbers and determine how to address that underlying issue. Also, since this is a setting that is often set when the user first installs or first runs the application, you'll need to determine if you should reconsider their initial decision, and when and how you will accomplish this.

If you already have usage data, there are several things that you can do to determine how representative your data is. For example, you can compare your sales figures to your dataset to determine if a given country is overrepresented in your usage data. To take another example, if you are collecting data about the hardware that your users are using, you could cross-reference it with data from another research method (survey, interviews, etc) to see if there is a discrepancy. There are certainly valid concerns about ensuring that the data that you collect is representative, especially if you are going to make decisions based on it.

While this is somewhat outside the scope of the original question, I feel that it's important to note that one must be careful in the decisions that are made based on anonymous usage data or statistical information. Presuming that the data is representative, all that you know is that something is happening (how frequently a given feature is being used, what the average screen resolution is, etc). You don't know why this is happening. Knowing the underlying reason is often more important than the raw usage or statistical information. A feature not being frequently used could be due to many different factors, including discoverability, usability, poor performance (real or perceived), invalid results, or that it's a feature that simply isn't needed frequently. If you learn via usage data that a feature isn't frequently used, then you need to understand why it isn't being frequently used.

3
  • This answer provide some very good perspective/insight into the issue. I don't necessarily think 1-3 are 'easy' to address because they deal with people who may think in various different ways, whereas 4-5 there is a more prescribed way to solve the problem. The point about understanding the underlying reason is very true, so I would say that capture some user information is just a starting point for working out some of the deeper issues.
    – Michael Lai
    Jul 16, 2013 at 0:39
  • I agree that they're not "easy", but rather that of the problems presented, they're the easiest to address within the application. The fourth problem is one that requires outside assistance (development has to figure out how to architect the system to allow for a plug-in that collects the data for a set period of time, legal has to engage with customers, etc) and is thus more difficult. The fifth problem is much much more difficult than the others.
    – nadyne
    Jul 16, 2013 at 0:42
  • Good points all around! #1 especially is very important to me as a user; I would much rather fill out a one-time (or even recurring) survey than give the okay to "anonymous usage data collection"...whatever that entails. And I'd always be worried whether it's really anonymous.
    – SilverWolf
    Oct 10, 2018 at 13:30
1

Show a one time dialog with a simple Yes/No option. Explain the situation in real terms, not just "help us out!" or "anonymous data". Briefly explain why your program should be allowed to collect that data, how it will most likely not effect the users.

For example: "Would you like to help us improve our app's performance and compatibility, by allowing us to collect information about your device hardware?" (I get that this example is a little technically wordy

Users are more likely to agree to something if they know a little bit about exactly what information they are giving up and how it really benefits you. The one danger here is making the explanation too long. The best approach might be a short snippet like above with a "Learn More" link for more concerned users.

See Steam Hardware Survey

2
  • 1
    So what is your experience with this approach? Did you find that you were actually able to get useful data from users?
    – Michael Lai
    Jul 15, 2013 at 22:25
  • @MichaelLai How useful the data is depends on what you collect and how you interpret it. Looking at something like Steam Hardware survey gives you an idea of what screen resolutions ram amounts etc, you feel comfortable targeting. The deltas every month also give you an idea of where the market is going.
    – user28446
    Jul 16, 2013 at 2:23
1

Provide an incentive to participate in this anonymous survey.

"Unlock the bonus feature by helping us improve. Anonymous survey time!"

Not sure about you, but I'd definitely do the survey for the bonus feature.

11
  • 1
    My feeling is that incentive based surveys might capture a specific group of participants and not be reflective of the entire population. However, most methods have similar issues to deal with. Is not helping to make the system better a good enough incentive for the users to contribute? Also, would they also be willing to do the survey again next time?
    – Michael Lai
    Jul 15, 2013 at 22:45
  • Who doesn't like immediate positive reinforcement? That is what incentives provide. That is why people spend spend spend, and go broke. Sad analogy, but it's true.
    – daCoda
    Jul 15, 2013 at 23:05
  • Well, most online marketing people will offer you a free ebook or something else to get your email address or contact detail, which they then sell to other people. Of course, the ultimate goal is to improve the product or service so you can sell more of it to the people. I guess I might feel offended if I am simply asked to help them improve the product which I am paying to use. I suppose no one would really consider needing an incentive if it was an opensource project instead?
    – Michael Lai
    Jul 15, 2013 at 23:14
  • I disagree with this approach. This sends the message that you don't respect privacy.
    – user28446
    Jul 16, 2013 at 2:21
  • @MichaelLai I agree - this is a good method for getting a lot of people to click a button, not provide a useful representation o f users.
    – user28446
    Jul 16, 2013 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.