Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

There are many different organizations starting to collect usage statistics in an attempt to improve the usability or user experience of their software. One of the common frustration when designing an application for a client or organization is that they have no idea of the typical screen resolution of their users, and therefore in most instances you have to resort to a 'fluid/response' design that doesn't necessarily cater very well for any screen sizes. Most of the statistics for display resolution comes from browser data, which doesn't necessarily reflect the organizations that use your application. Therefore, I think it is quite important for software to collect usage statistics to help improve all aspects of the design.

However, when a screen prompt asks you to consent to anonymous user data collection, I think many people would probably not participate even though collectively it may be in the best interest to do so. Are there any ideas or examples where anonymous user statistics reporting rate can be improved, whether it is through the design of the interface or a better wording?

The Steam/Valve Hardware and Software survey in one of the answers provided is a very good example of making the user data work hard to improve the service. Interesting enough I haven't many other examples around, so if you have seen something along these lines please also feel free to share it.

share|improve this question
    
The problematic part is not only privacy, but also the unknown systematic bias in use patterns between responders and non-responders. Thus wide surveys "on the cheap" are often worse than controlled studies. –  Deer Hunter Jul 15 '13 at 14:57
    
Agreed, so how can we make it better? Or should we be using it at all? –  Michael Lai Jul 15 '13 at 22:24

3 Answers 3

up vote 2 down vote accepted

I think that this question is multi-faceted, and that there is no single answer to it. On a basic level, I think it's important to remember that the easiest and safest course of action for the user is to say "no". To get them to say "yes" (and I strongly believe that giving such information needs to be opt-in and not opt-out), there are many things that must be addressed.

In my experience, it is true that many people don't participate in the collection of anonymous usage data. I have worked on different products with different user bases. The levels of participation in each of those are proprietary information and thus I can't share them, but it's fair to say that many people don't participate. There are many different and valid reasons to choose not to participate:

  1. I don't know what data is being gathered and what is being done with it.

  2. I do not understand what the personal benefit is to me to allow this information to be collected.

  3. I am concerned whether I will experience any kind of negative effect as a result of participating in such a program, such as reduced download speeds when the program uploads my data.

  4. If I'm using a corporate device or network, my employer might not allow (or might actively block) such data from being shared. Alternately, I might not be sure whether my employer allows it and I don't see a very good reason for me to research an answer to this question.

  5. I have to decide whether I think that the company/project/whatever that is gathering the information is trustworthy.

Problems 1-3 are the easiest to address, and are the ones that are most amenable to being addressed in an individual application. The fourth problem is somewhat more difficult to address, but is probably addressable with some hard work and creativity. The fifth problem is likely the most difficult to address, and is probably outside the scope of the work that a designer does on an individual application.

If you are going to deploy an application that collects anonymous usage data, then you need to understand which of these items are the ones that are the most impactful for your users. For example, if one were working on enterprise software, companies that want to ensure that none of their proprietary information is shared inadvertently. One method to address this concern might be to work with an individual customer to gather this data for a set period of time, and to give them the opportunity to check the data that is gathered before it is shared with you. Repeating this process with multiple customers will give you a breadth of data.

If you already have an application that collects anonymous usage data and you feel that you can improve the rate of participation, then you need to determine what impacts your low numbers and determine how to address that underlying issue. Also, since this is a setting that is often set when the user first installs or first runs the application, you'll need to determine if you should reconsider their initial decision, and when and how you will accomplish this.

If you already have usage data, there are several things that you can do to determine how representative your data is. For example, you can compare your sales figures to your dataset to determine if a given country is overrepresented in your usage data. To take another example, if you are collecting data about the hardware that your users are using, you could cross-reference it with data from another research method (survey, interviews, etc) to see if there is a discrepancy. There are certainly valid concerns about ensuring that the data that you collect is representative, especially if you are going to make decisions based on it.

While this is somewhat outside the scope of the original question, I feel that it's important to note that one must be careful in the decisions that are made based on anonymous usage data or statistical information. Presuming that the data is representative, all that you know is that something is happening (how frequently a given feature is being used, what the average screen resolution is, etc). You don't know why this is happening. Knowing the underlying reason is often more important than the raw usage or statistical information. A feature not being frequently used could be due to many different factors, including discoverability, usability, poor performance (real or perceived), invalid results, or that it's a feature that simply isn't needed frequently. If you learn via usage data that a feature isn't frequently used, then you need to understand why it isn't being frequently used.

share|improve this answer
    
This answer provide some very good perspective/insight into the issue. I don't necessarily think 1-3 are 'easy' to address because they deal with people who may think in various different ways, whereas 4-5 there is a more prescribed way to solve the problem. The point about understanding the underlying reason is very true, so I would say that capture some user information is just a starting point for working out some of the deeper issues. –  Michael Lai Jul 16 '13 at 0:39
    
I agree that they're not "easy", but rather that of the problems presented, they're the easiest to address within the application. The fourth problem is one that requires outside assistance (development has to figure out how to architect the system to allow for a plug-in that collects the data for a set period of time, legal has to engage with customers, etc) and is thus more difficult. The fifth problem is much much more difficult than the others. –  nadyne Jul 16 '13 at 0:42

Show a one time dialog with a simple Yes/No option. Explain the situation in real terms, not just "help us out!" or "anonymous data". Briefly explain why your program should be allowed to collect that data, how it will most likely not effect the users.

For example: "Would you like to help us improve our app's performance and compatibility, by allowing us to collect information about your device hardware?" (I get that this example is a little technically wordy

Users are more likely to agree to something if they know a little bit about exactly what information they are giving up and how it really benefits you. The one danger here is making the explanation too long. The best approach might be a short snippet like above with a "Learn More" link for more concerned users.

See Steam Hardware Survey

share|improve this answer
1  
So what is your experience with this approach? Did you find that you were actually able to get useful data from users? –  Michael Lai Jul 15 '13 at 22:25
    
@MichaelLai How useful the data is depends on what you collect and how you interpret it. Looking at something like Steam Hardware survey gives you an idea of what screen resolutions ram amounts etc, you feel comfortable targeting. The deltas every month also give you an idea of where the market is going. –  Justin Meiners Jul 16 '13 at 2:23

Provide an incentive to participate in this anonymous survey.

"Unlock the bonus feature by helping us improve. Anonymous survey time!"

Not sure about you, but I'd definitely do the survey for the bonus feature.

share|improve this answer
1  
My feeling is that incentive based surveys might capture a specific group of participants and not be reflective of the entire population. However, most methods have similar issues to deal with. Is not helping to make the system better a good enough incentive for the users to contribute? Also, would they also be willing to do the survey again next time? –  Michael Lai Jul 15 '13 at 22:45
    
Who doesn't like immediate positive reinforcement? That is what incentives provide. That is why people spend spend spend, and go broke. Sad analogy, but it's true. –  daCoda Jul 15 '13 at 23:05
    
Well, most online marketing people will offer you a free ebook or something else to get your email address or contact detail, which they then sell to other people. Of course, the ultimate goal is to improve the product or service so you can sell more of it to the people. I guess I might feel offended if I am simply asked to help them improve the product which I am paying to use. I suppose no one would really consider needing an incentive if it was an opensource project instead? –  Michael Lai Jul 15 '13 at 23:14
    
I disagree with this approach. This sends the message that you don't respect privacy. –  Justin Meiners Jul 16 '13 at 2:21
    
@MichaelLai I agree - this is a good method for getting a lot of people to click a button, not provide a useful representation o f users. –  Justin Meiners Jul 16 '13 at 2:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.