Skip to main content
20 votes

A/B testing - client wanted the test run 70/30

Your statistical power is maximized if equal numbers of users are directed to A and B. That is, assuming one design really is better than the other, you’ll achieve statistical significance with the ...
Michael Zuschlag's user avatar
4 votes

A/B testing - client wanted the test run 70/30

In theory, there isn't a problem with a client-ordered 70/30 test, as long as you're comparing conversion rates and not conversion amounts. If your hypothesis holds up, you'll still see the 5% dip ...
Izquierdo's user avatar
  • 12.7k
4 votes

A/B testing - client wanted the test run 70/30

Run A/B test on 60% of the traffic then. Diverged exactly 50/50 into old and new variants. Diverge all other 40% of users/sessions/views to new experience (as your client requests), but ignore them in ...
Frax's user avatar
  • 141
2 votes

A/B testing - client wanted the test run 70/30

Your customer wants two things: Find out with evidence which method gives better sales, and don't lose sales during the test. A 50/50 split will give you the evidence quickest. A 70/30 split ...
gnasher729's user avatar
1 vote
Accepted

Explain difference, and timeline of Pilot test vs Beta test?

The difference is actually really simple: Pilot testing is done in a controlled environment (like your own company) where you can study the product design and participants in a really controlled ...
Devin's user avatar
  • 38.1k
1 vote

What's the difference between third-party surveys and follow-up questionnaires?

The main differences that I know of from my experience using both: Some third party apps are able capture the results as soon as the user clicks on an option - and do not need to press 'submit' etc. ...
Kish's user avatar
  • 2,607

Only top scored, non community-wiki answers of a minimum length are eligible