Usability or Content: Knowing which is the cause of the problem?

Question

I've been spending time looking at pages and page flows trying to reduce bail rates and increase completion rates on large ecomm site when a brand manager asked a very intriguing question, "How do you know when it is no longer the page that is causing people to leave but instead it is the content that is the issue?"

I had some initial thoughts and theories that I shared but the question really got me thinking, how do we know and can we make it quantitative?

Has anyone experienced anything similar? If so, how was it handled?

Any ideas for testing something like this?

Answer 1

A good way to test if content is the problem is A/B testing. If you are dealing with a product page, have two different versions of the product page written up by the content developers, then split the users in half - each viewing only one of the content pages, and compare the completion rates for the two groups. If there is significant variance in completion rates, and the new content is getting more completions, then it could very well be the content that is causing the problem.

I would recommend doing this for the top 10 products or so, or the top 10 products where completion rates are a concern. Then, if it is successful, expand it to additional products.

As for how to change the content, I couldn't really tell you without seeing an example. It could just be formatting the content differently, being more concise with the content there, or it could be completely rewriting the copy. Honestly, you will probably need to try out different things to find out how it works.

Answer 2

You may want to consider utilizing Google analytics "custom events", to determine if content is the reason users are dropping off.

You can use these custom events to set off triggers that will feedback to you whether the user has begun reading, abandoned after starting to read, or finishes reading the entire article/text.

I must warn you however, that this perhaps isn't the most reliable method of determining a reason for abandonment, but it certainly gives you actionable metrics for you to work with.

Justin Cutroni put together a great tutorial on how to implement this

Answer 3

Having been in the same situation I understood that one of the biggest things with e-commerce is that it is easy to see what happens (analytics), but not why. The why has two sources:

1. Usability issues
   • A user doesn't understand how to perform something they want
2. Psychological reasons
   • A user is not sure they want to do something even if it is usable

It is important to understand that no flow will have 100% conversion rates. The statistics can be improved with usability testing, A/B testing and design iterations. When you push the envelope to convert more and more people you get into a shady area of possibly starting to manipulate users, which is detrimental to both the company and the consumer.

Answer 4

One could argue that if people saw the content (if it was clearly presented) and bailed then there would be no further interaction on the page and the problem is probably the content. On the other hand, if it looked like people were trying to find information on the page then you can't rule out usability or content. It depends on the amount of data that you have, and also if there are similar pages that you can compare. Ideally you will have access to users and analytic information to help you understand the general user interactions on the page, but if not then even doing some simple usability testing on some public sites can give you general ideas on whether it is the usability or the content.