4

Hello I'm quite newbie in UX, especially in A/B Testing methods.

In my opinion and after I read some of articles. A/B testing usually used for analyze some new call to action button, copywriting, layout, or colors. Measure the new and existing design with conversion rate, etc.

But the question is, can A/B testing used for analyze new features? For the example I have 2 variation of design. existing and new design. I want to implement new features in new design. So, I doing A/B testing for monitoring the conversion rate of:

  • Existing design (without new feature).
  • New design (with new feature).

Is it possible to find a winner of that variations? Can we conclude that our new feature have a positive response for users?

Improve this question
2
  • 1
    It depends. If you can't test the new design without the new feature then go for it. If you can you should test the design and the feature separately. Otherwise it is hard to tell how much the feature and the design are responsible for the result.
    – jazZRo
    Feb 15, 2021 at 10:07
  • You can use testing to prove or disprove anything, but the validity of the test depends on how objective you can be with the hypothesis proposed and how the results are interpreted. As with all types of research, it is important not to assume that the answer that you find is the right one, but to keep testing assumptions and validate them.
    – Michael Lai ♦
    Feb 16, 2021 at 0:57

1 Answer 1

Reset to default
2

Yes. This is done in the industry, often as a rollout plan when releasing a new design of a major feature.

For example the new design of the feature will be revealed to 10% of the audience for 2 weeks, and the team would observe user behaviour for the new feature via analytics and user feedback. If the response is positive they would increase the % of the user base the new feature is revealed to. e.g to 20%, 50% until 100% of the user base is using the new version and what used to be the existing version is no longer in use.

This is because the team would like to assess how real users accept the new design without risking any loss of conversions or suffer from bad user experience. Even tho user testing may have been done in the design process, there is a possibility of unexpected hiccups to occur in the production live environment of the app/website.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.