1

The test is to trace CTR.

Only variation has button so the test results as below:

Original - Visitors: 120170, Conversion rate: 0

Variation - Visitors: 121058, Conversion rate: 15256

Next step is to evaluate the test results, now I wonder what measurement can i use to know whether 15256 is consider high or low (significant or not?), should we implement this button in the system? It would be highly appreciate to listen some feedback from expert who has any background or experience on this.

Thanks.

Improve this question
1
  • Hi Sonia, and welcome to the UX site. It would be useful if your answer had a bit more background: what kind of user base do you have, and what kind of action are you asking them to perform?
    – Vince Bowdren
    Aug 15, 2016 at 13:35

1 Answer 1

Reset to default
0

High or low compared to what?

Your conversion rate is 12.6%, but this figure on its own is meaningless. So you need to compare it to something. That something will depend on what your solution is, e.g. is there an industry benchmark for the industry sector your solution works in?

The Click Through Rate is usually used by ad companies who want to see how well their ad attracts visitors and just measures the ratio of clicks to impressions of your page, email or advertisement, however clicking doesn't necessarily correlate with conversion.

Obviously looking at your two examples, the variation with the button is the clear winner because the original had no conversions.

The problem with a lot of these metrics like Click Through Rate (CTR) is they don't tell you 'why' someone clicked nor how they feel. Answering why questions is a User Experience thing, and there are much better metrics for measuring the user experience than CTR which is a marketing metric. People don't click ads in the same way few people watching television don't watch commercials.

https://www.aabacosmallbusiness.com/advisor/important-click-rate-ctr-really-042603295.html

Improve this answer
4
  • Thanks. We just try to find out whether it is worthy to put a button there by running a AB test to keep track of CTR of that button, although we know that it is definitely winner for original, simply to find out how many visitors would really mind to click on the button on a flight select page.
    – Sonia Yunni
    Aug 15, 2016 at 10:00
  • Like any metric, it has to be aligned with some goal, e.g. to improve something, or to reduce something. It is fine to start tracking the CTR of the new button to ascertain a benchmark in preparation of some future A/B test, e.g. maybe you will vary to position or styling of this new button in the future so you will need the current benchmark to compare to.
    – SteveD
    Aug 15, 2016 at 10:08
  • From your experience, how do you think I can setup the test for a scenario like this? Just to know if a close button is important to be included as now we are taking page drop off as the metric to measure. Reason being to take page drop off to measure because we want to speed up user buying process. Any advice?
    – Sonia Yunni
    Aug 19, 2016 at 9:43
  • You want to understand why people are dropping off, which probably means you are going to need to talk to some of them. Google analytics can only tell you so much (they might tell you where people drop off), but they wont tell you why people are dropping off, and you will only get that answer by either speaking to them or watching them. Remember that there are valid reasons for drop off, i.e. they simply weren't interested, as well as usability reasons, i.e. your website is asking for too much or is too confusing to use.
    – SteveD
    Aug 19, 2016 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.