Let's say I have a program with an automatic-updating feature. When I'm shipping a new update, I can do either of the following (or both):
- Provide a beta channel for advanced users to test the latest updates before deploying them to everyone (the traditional approach), or
- Roll out updates in a staged manner, gradually over random subsets of users until the update hits everyone (the Google approach).
The first approach gives technically-inclined users a chance to try out the new version ahead of everyone else, but expects the user to understand that there may be problems and that they have a responsibility to report them. Because beta versions are typically sampled by power users, this can lead to inadequate coverage of certain system configurations.
The second approach, pioneered by Google, ships the update to random subsets of users over time until everyone gets the update. This approach is much more likely to sample the target audience evenly and properly cover the myriad configurations users have. It can also be stopped at the first sign of trouble. However, it could leave some users waiting longer than expected for a new version to reach them, and a bad update could negatively impact a subset of users who aren't knowledgeable enough to report the problems as bugs or understand the staged rollout process.
How do I choose between these two approaches (or a combination of both)? What other approaches should I consider?