Is it correct to perform A/B testing for 3 versions. It may sound funny as the title of test answers the question, but still is there any practice to test 3 versions against each other.

Thank you in advance.

  • 2
    If you are referring to three whole page versions then yes, that's fine and is still an A/B test (or possibly known as ABn test). But if you're referring to multiple variants of elements on the same page, then no, that would be multi-variant testing. – JonW Nov 25 '14 at 14:38
  • Yes, there are 3 whole page versions. – Armine Nov 25 '14 at 14:46
  • After some research I found a doc describing:"When A/B testing is used in this way, a third or even fourth version of the page is often included in the test, which is sometimes called an A/B/C/D test" – Armine Nov 25 '14 at 14:46
  • As long as you have enough traffic to allow to tests to reach high enough accuracy, it should not be a problem at all. – Daniel Zahra Nov 25 '14 at 16:02

Sure! Why not?

After all, you need to figure out what's best for your product in your process. If you have three alternatives, then go ahead and test those three alternatives. "A/B testing" is just a name. Not a law.

Now, there are a few things you should be aware of when you collect data.
One of the problems with A/B-tests is that you need a lot of data and you need very convincing data. Using 3 designs instead of 2 designs might affect this. (Eg. 50/30/20 instead of 80/20).

Also remember that you should look at A/B tests as an ongoing opportunity to measure certain aspects of your solution. Amazon, Google, and all the big companies who are well known for their successful A/B-research, they conduct continuously A/B-research.

If you find the numbers hard to analyze, and if you think it's difficult to draw any clear conclusion from the A/B/C-test results, then you might be better off with some traditional user-testing where you can get additional feedback from the user.



This also called multi-variate testing and the method differs slightly.

The addition of another page test component adds the potential for more confounds and misplaced findings though. If possible it's best to stick with a/b and just run another round.

Here's some more reading: http://blog.hubspot.com/blog/tabid/6307/bid/30556/The-Critical-Difference-Between-A-B-and-Multivariate-Tests.aspx

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.