A way to test the UX of an application by displaying one version of it to 50% of the users of the application, and displaying another version to the rest of the users. You then compare how the two versions performed by using metrics such as click through rates, the average time on the site, etc.

learn more… | top users | synonyms

1
vote
2answers
48 views

Direction of fretboard

I know it might be a bit difficult to find musicians who have experience with UX, but I'll give my question a try here. I am designing an app for musicians and need an opinion on something: Which ...
7
votes
4answers
1k views

How does e-commerce product quick view affect conversion rates?

Many large e-commerce retailers are using "quick views" for products on their collection pages where more details appear either on-hover of via some other trigger. Has anyone performed any A/B tests ...
9
votes
4answers
174 views

A/B Testing. How should it be done?

I just stumbled upon Google Website Optimizer. Is this the way to go? How do you normaly do A/B tests with your website?
2
votes
5answers
165 views

A/B testing in user test

I designed 4 versions of a checkout page. I want to make some user test to know which version performs better. Do you think that I have to do the tests with the same person? the big change in the ...
6
votes
10answers
549 views

Isn't all user testing essentially a way of gathering how people think things should work?

When user testing is performed, users are often presented with a serious of variations of a product and observed to see where they excelled and where they had trouble. People will often click in areas ...
2
votes
2answers
539 views

Increasing add-to-cart rate, decreasing conversion?

I am currently optimising an e-commerce website. The request was specific to the product pages, after gaining insight from analytics and feedback two alternatives were proposed, and tested. The ...
1
vote
3answers
136 views

For landing page A/B testing, how do you recruit from a niche market?

I'm currently working with a client that has not launched the full website yet, but is A/B testing ads to drive member acquisition. The ads direct users to a landing page that collects user e-mails. ...
4
votes
5answers
407 views

Which spot has the best conversion in the grid layout?

We are going to show popup with 6 different offers, positioned by 3 in two rows. Of the these 6 offers, 2 are really beneficial for us (say, have high reward for click). Common sense says that we ...
5
votes
1answer
118 views

Conducting different tests on a website at once

We have our own A/B test software which works this way: Every time a user visits a page with a test, he gets a cookie marker. For variant A in test one it may look like this: test1_a For test 2 and ...
0
votes
2answers
75 views

How to test for the best colour of an online store?

I already ran a test set up as following: I told the users what they were going to buy, showed them the same webpage in eight different colours (four webpages on two rows), and asked them on which ...
6
votes
4answers
1k views

How to A/B test a published IOS and Android app on actual users?

I've published several apps on IOS and Android adding flurry to measure the interaction that is happening in those apps, however, I'd love to enhance this interaction and make my apps more useful by ...
3
votes
0answers
57 views

How can I learn A/B testing independently? [closed]

I'm looking for an independent project to learn A/B testing on, and I'm trying to figure out which A/B testing tools are commonly used for: Mobile Games Web And I'm also aware that the common ...
14
votes
2answers
231 views

How to realistically test blog headlines before publishing?

I'm currently writing a post for my company's blog. I'm agonising over the headline, as I really, really want people to read this post (I'm trying to convince our users to opt in to our customer ...
-1
votes
3answers
119 views

A/B or multi-variate testing as replacement for UX design?

It seems like in a business-driven organization (i.e. where the sales and marketing team has a bigger say) that offer software products or services, A/B testing is used as a way to improve and measure ...
4
votes
1answer
205 views

A/B Testing Home Page of an Ecommerce Site

I am very new to any sort of website testing (please be gentle!), and I have some questions (mostly just for my own understanding) about A/B testing an ecommerce site's home page. The company I work ...
6
votes
1answer
706 views

A/B Testing vs Cohort Analysis

I had a few questions about A/B Testing vs Cohort Analysis (dividing users based on registration time). 1.) Is A/B testing mainly used for UX design changes (such as change in Sign Up button Color) ...
12
votes
4answers
493 views

A/B Testing Technique

My friend and I were having a discussion about different ways of A/B testing. His technique was to make a single small change (e.g. changing the colour of a button) and then perform A/B Testing to see ...
6
votes
1answer
304 views

A/B testing a site with low traffic

I've always been very interested in A/B testing, but have never had the opportunity to implement it on any websites. I work at a digital agency where the main reason why we haven't been able to ...
6
votes
2answers
333 views

Unexpected A/B testing results

We've just run an A/B test for one of our landing pages using Visual Website Optimizer. We are an online hotel booking reservation site. The landing page shows information about a particular hotel ...
1
vote
2answers
257 views

What other solutions do you use besides A/B tests for testing the efficiency of two site structures?

I've designed new site structures for one of our clients, and want to test which brings better conversion rates. using A/B testing seems obvious but are there any other solutions to cite ...
12
votes
5answers
322 views

Tips for A/B testing?

There seems to an increasing number of A/B testing services (such as Google Website Optimiser) and people using them. Have you got any tips on using and analysing the results from them?
2
votes
2answers
83 views

What metric should I use for A/B testing of a blog homepage?

We are launching a blog about web design. The goal of our blog is of an ordinary content site - to get visitors to read as many articles as possible (NOT to get ad impressions). Our designer has ...
6
votes
2answers
685 views

A/B testing for mobile apps? [duplicate]

Possible Duplicate: How to A/B test a published IOS and Android app on actual users? Is there any proper way and tool to do A/B testing in mobile apps to find the users' engagement and pain ...
7
votes
2answers
282 views

A/B test shows unexplainable winner for a remote goal

We ran a test where we tested 3 different button colors for a Call to Action. The CTA led to our offers page (where we had special offers to sign up for a TV package). But of the 3 variations, none ...
1
vote
4answers
210 views

Who should be responsible for A/B testing and conversion optimization?

Currently, I'm wearing many hats, but looking to expand our team and eventually dedicate someone to running A/B, MVT and conversion optimizations for us. With companies where even minor % ...
1
vote
1answer
100 views

I have a few significantly different layouts I want to test, is A/B or MVT testing the best approach?

I've read that A/B testing and MVT are best for testing smaller scale changes (i.e. button color, headline copy, etc), but are they also suitable for testing two (or more) very different designs? ...
1
vote
1answer
149 views

A/B Testing and Duplicate Content [closed]

Now that Google can crawl/index content from JavaScript, how are page variants treated by Google by serving a slightly different versions when A/B testing? Is it advisable to add references to the ...
3
votes
4answers
373 views

Testing OK/Cancel buttons

Related: OK/Cancel on left/right? I have been looking in to whether OK/Cancel should be on left or right, and after looking at lots of views on each side it seems there are good reasons for both. I ...
7
votes
1answer
165 views

What do you do with an inconclusive AB Test? Keep the change or lose it?

Short Version If you run an AB Test, and the results are that there's no difference in your key metric, do you keep the experimental version, or revert the site back to the control? Story Version ...
2
votes
1answer
105 views

Multivariate test conversion: short vs long path

Let's assume we a workflow with 3 steps. Page 1 is our landing page. Page 2 is the form (signup). Page 3 is the conversion page (thank you page). To test variations on page 1, I am running a ...
14
votes
2answers
2k views

How much do “where did you hear about us?” type questions impact signup/conversion rates?

There's a truism I often hear - claiming that the less complicated the signup process for a web-service, the higher the conversion rate (as a rough measure of complexity, count mouse clicks to signup ...
5
votes
3answers
278 views

Better purchase conversions for single page vs. multiple?

I'm mocking up a page for purchasing our product, and I'm wondering is it better to do everything on one page or split the ordering and payment into two different pages? Anyone have empirical data on ...
1
vote
1answer
127 views

Is there any testing methodology similar to A/B testing, but without a specific goal?

Please forgive me; this may be somewhat long-winded. I work in an academic library. I'd like to study how well assorted features of our site are serving our users' needs; for example, labeling of UI ...
4
votes
5answers
313 views

A/B Testing Homepage Variants

When trying out new homepage layouts / styles, is A/B testing the best way to do this? With typical A/B tests you have a specific conversion measurement (i.e. version X drives significantly more ...
3
votes
1answer
171 views

Which people do I choose for large scale A/B testing?

What are the best practices for deciding which users to admit to a large scale A/B test on a website? Below are two of my concerns on this choosing process. Veteran users' aversion to change Humans ...
3
votes
5answers
425 views

Is A/B testing a worthwhile use of resources for the end user of a mobile app?

Web sites can enable A/B testing by serving a different page, for example. A mobile app would have to collect data and return it. As a matter of principle, is it okay to have software "phone home" ...
6
votes
2answers
223 views

Are there any easy to use A/B Testing frameworks?

How to do A/B testing is a difficult problem. Are there any frameworks or tools out there that ease the pain?
7
votes
5answers
281 views

What are the best A/B tools? [closed]

I have a few pages to A/B test, and I am looking for the best tools to use for this? Is there a "standard" or favorite industry tool?
7
votes
1answer
264 views

Are simultaneous tests on different stages of a multi-step form statistically independent?

I want to do some test-and-learn on a multi-step form using infrastructure which appears to support conducting and reporting of A/B/n tests. If my site had two different forms I could simultaneously ...