I just stumbled upon Google Website Optimizer. Is this the way to go?
How do you normaly do A/B tests with your website?
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.Sign up to join this community
I used to work at Intuit and they tested everything. Here are some lessons I learned:
Also, there are other kinds of tests. Satisfaction surveys being one example. But a website has lots of goals besides those that can be measured on Google. For instance, Ask people who visit the site: "What does this company do?" And see what they say. Test different messaging. Which gives the most accurate rendition.
Bottom line: (imho) A/B testing is only useful in high traffic controlled situations on e-commerce (or highly "goal oriented") sites. A standard website, you are better off with hallway usability tests.
For a quick intro there's a good video tutorial on YouTube entitled "Planning and Running your First Experiment with Google Website Optimizer". It's over 1h long but well worth your time if you're new to GWE and I think it answers your question.
GWE is focused on testing for improved conversion rates. Would be great to know your opinions if increased conversion is always related to better usability.
I'd recommend reading Always be testing (Amazon.com) which is especially written for Google Website Optimzer. It is aimed at beginners and does a very good job of introducing the subject and giving ideas to begin testing with.