We have a product that goes behind the firewall and frequently pings back a (small) amount of information once a week or less. While we have tossed around a number of ideas:
- Paying the customer to let us slurp down user engagement metrics on a monthly (or more) basis
- Reporting back one or more data points during update checking (with the customer's permission, of course)
- JUST reporting back crashes and attaching some information about session to that (again, with the customer's permission)
- Giving up and just doing weekly-ish visits with customers and walking them through situation to get feedback
None feel like the best practice. Our goal is to achieve the Etsy-style [1] continuous experimentation and, while that will never be the case, we'd love to get closer. Any best practices we should look at?
[1] http://apptimize.com/blog/2014/01/etsy-continuous-innovation-ab-testing/