We are going to present the option to try the Beta version of our product to a sample population but we don't know how to calculate to how many of our users should we release it? Should I use the same sample population calculation that we use for surveys? I was thinking of using this calculator based on our total population (we have this information since this is a redesign) https://www.qualtrics.com/blog/calculating-sample-size/ Is it correct that I use this calculation to define our Beta users?

How do you calculate to how many users to release your Beta version so they can try it out?

Thank you in advance,

  • 1
    I suspect that it may help to know roughly what sort of numbers you're talking about: do your existing users number in the tens, hundreds, thousands or millions? I also suspect a lot will depend on how much resource you have to handle beta feedback (you are likely to need more resource than just processing survey results).
    – TripeHound
    Commented Sep 12, 2018 at 8:19
  • We have thousands of users in our total population. Commented Sep 12, 2018 at 14:01

3 Answers 3


Why are you talking hard numbers ?

The whole point of the Beta program is to get users to test a new version, you want them to test the program extensively and report any issues. Ideally, you would want the techier tier of your user base, because they have more experience in finding bugs, that are also fanboys, as fanboys are eager to help squash any bugs found.

Enrolling users on a Beta program because they took the time to answer a survey is, I think, getting the target population wrong.

Do as many other software companies do, offer a Beta version alongside the usual version with a message reading that the Beta version is a development version that needs to be tested, not for the faint of heart.

  • Thanks. Yes, that is what we are doing: we are releasing a beta version alongside the usual version with a message letting them try the new beta if they want to. But we dont know to how many users we should show that opt-in message. Commented Sep 15, 2018 at 18:43

It depends on what kind of research questions you have and what kind of data you want to collect.

The Qualtrics tool is great for any kind of quantitative analysis. Think of the margin of error as a kind of sensitivity tool: if there is an issue that affects 5% of users or less, you won't be able to detect it for sure if you have a margin of error of 5%.

If you're interested in qualitative feedback, then having a sample of this size will likely generate a large amount of text to read and analyze. If you can set up a short survey to get structured feedback, that will be helpful.


There is not a certain number of testers. You must determine the goals of your analysis. Also you must determine the planned number of users of your product. After that you can understand what types of testers you need, how many types and how many testers you need.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.