2

We have a product that goes behind the firewall and frequently pings back a (small) amount of information once a week or less. While we have tossed around a number of ideas:

  • Paying the customer to let us slurp down user engagement metrics on a monthly (or more) basis
  • Reporting back one or more data points during update checking (with the customer's permission, of course)
  • JUST reporting back crashes and attaching some information about session to that (again, with the customer's permission)
  • Giving up and just doing weekly-ish visits with customers and walking them through situation to get feedback

None feel like the best practice. Our goal is to achieve the Etsy-style [1] continuous experimentation and, while that will never be the case, we'd love to get closer. Any best practices we should look at?

[1] http://apptimize.com/blog/2014/01/etsy-continuous-innovation-ab-testing/

Improve this question
4
  • The more data UX can collect, the more you can do with it. But it's hard to say which of these would be best for you. It would all depend on a lot of factors.
    – DA01
    Jan 16, 2015 at 21:48
  • Definitely - as I mentioned, I think we'd love to start with the basics of what is currently provided to Web apps (Clicky, GA, MixPanel, etc) - just seeing what's used and when. The problem is that all these are designed to phone home regularly, and we can't.
    – aronchick
    Jan 16, 2015 at 21:49
  • 1
    I'd maybe suggest start by talking to your customers. What would they consider preferable?
    – DA01
    Jan 16, 2015 at 21:53
  • It's a great idea, but I'd like to bring a set of best practices to them to help us choose between. Some boiled down list of the above is close, but I wanted to make sure I wasn't missing anything.
    – aronchick
    Jan 18, 2015 at 19:26

1 Answer 1

Reset to default
1

Ahh I feel your pain. This word may be your product's greatest strength and yet its greatest weakness -> Enterprise.

I can realise where your customer is coming from. The data you want to collect is pretty harmless for regular consumers as long as its anonymized. But for enterprises, you are asking them to share how they operate your software potentially leaking Secretive Business Practice! We faced a similar situation when our product was being used by a Government Entity and any sort of analytics information meant Tax Default data of Fortune 500 companies!!!

You could talk to your customer but do remember a key point. The person in your client organisation with whom you have a relationship with is almost in all cases a non user of your software. This can be a C-Level Executive or the Purchasing Head or Dept. Head etc. So to be effective you have to break channels and heirarchy, you have to be talking to Data Entry operators / Clerks / Engineers who actually use it. This could work out in your favour as they would probably ignore the above point. Your sales team would have to reach out till them though. We had a chat with them during training sessions and reviews and we got very very useful feedback.

Incase your customer doesn't mind sharing data. I wouldn't advice paying them. You are doing this after all to improve your product's usability which in turn is going to increase your client's efficiency.

My suggestion to you would be to not collect analytics data for your app in production but to have a separate team / set of consultants use your product the way your end customers would and collect analytics data from there. It may not be 100% accurate data but it is certainly data which you can use.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.