Why did (ecommerce) registration rate go down at checkout when we tested social login buttons?

Question

I work for a small ecommerce site and a bit ago we tested social login buttons for account creation.

The way our site usually is setup is after checkout if you're not logged in, we mention you can save all your billing/shipping details if you just drop in a password and click "go."

For a sample period of time we did a test where we had a "create with Facebook" buttons on the page above that password box.

While the buttons didn't work (a message came up explaining we were testing it, and you needed to use the box below) we counted a click on that button as a "successful" registration for the sake of the test.

At the end of the test period we saw dramatic increases in the registration rates on our "add to wish list" user flow, but on the confirmation page we saw a big drop off in registration.

The day we took the dummy buttons off, the rate went back to normal. I pulled numbers for a month prior, 2 weeks prior and 1 week after as different benchmarks and in each case the registration on that confirmation page is much worse during our test period.

Why are people clicking the "login" or "create with facebook" on the wish list flow but on the order confirmation page it's making less people register.

TLDR: tested out "create account with FB" button on order confirmation page and registration dropped in 1/2 compared to benchmark periods with simple form...why?

I'm helping give a presentation next week about it at my company so any advice/direction on this would be great.

Answer 1

It's taken me a while, but I've considered all the info you've provided and have a possible answer for you. First I want to summarise my understanding of what you've done, just to make sure we're on the same page.

In summary, you measured the rate of user registrations on both your wish list and order confirmation pages for a period of some weeks. You've then broken up those results into three main groups:

1. Prior to the test;
2. During the test; and,
3. Following the test.

You found that the rate of registrations for the periods prior to and following the test period was almost on average double that during the test period. The only difference, as far as you can see, is that during the test period you provided the option for users to use either their Facebook or Google credentials to login.

Now, I know I asked you a lot of questions in my comments, but what I was trying to do was get a feel for the respective user interfaces and workflows between the 'wish list' and 'order confirmation' pages. I wanted to understand why there may have been a difference specifically between these two, rather than an overall difference between the level of registrations with and without the social media login options.

Unfortunately, without screenshots of the actual user interfaces or the ability to go and visit the site in question and 'try it out' myself, I have an answer that will hopefully shed some light on your problem, but may not answer it entirely. Hopefully it helps with your presentation!

So-called 'social media login buttons' have now been around for five or so years. It wasn't long before these options took off and became evermore prevalent in websites from all sorts of fields. This seemed to coincide with a marked improvement in login rates and a reduction in failed login attempts.

Some of the drivers towards using social login buttons were:

• as many as 86% of users were put off by having to create new accounts on websites (see http://www.webhostingbuzz.com/blog/2013/03/21/whos-sharing-what/)
• over 50% of users were tempted to leave a site rather than having to complete another registration (see http://www.prweb.com/releases/2012/1/prweb9086226.htm)
• 77% of users believed that social login was a good registration solution (again, refer to http://www.webhostingbuzz.com/blog/2013/03/21/whos-sharing-what/)
• a third of users frequently choose to leave a site rather than resetting their login info when they've forgotten it (see http://www1.janrain.com/rs/janrain/images/Industry-Research-Value-of-Social-Login-2013.pdf)

However, while the above may have all been true (and, to some extent still is), something changed in June 2013. Namely, the revelation of classified NSA documents by Edward Snowden opened the eyes of a lot of people in terms of their privacy. This was only the beginning, as a number of events have since occured that have raised the ire of internet users (e.g. revenge porn sites, the so-called 'fappening', the highly targeted nature of online advertising, suspicious terms & conditions associated with the use of certain websites/apps, etc etc). This has all culminated to the embracing by some of the largest IT companies to protect the privacy of its users (e.g. Apple's allowance of ad blockers since iOS 9, Apple's current fight with the FBI, etc).

As a result, what we have today is a somewhat more sceptical and aware audience from which to attract our user base. And increasingly we're seeing 'security experts' and a range of 'bloggers' arguing against the use of social media login buttons with arguments like:

• You are weakening the security of your Facebook, Google, etc accounts by reusing these across many multiple sites.
• You are weakening the security of the site you are logging into with your Facebook, Google, etc credentials.
• You are tying another identity to what may be unrelated or conflicting activities.
• You are adding to the pool of 'public' information about you.

However, the most relevant factor to your scenario is that more and more people are recommending that users "avoid using social media sign ons for ecommerce sites". While this type of advice may not yet be resonating with a majority of users, it is becoming more prevalent.

I suspect that in your instance, when a user got to the stage of an actual order 'confirmation', they were hesitating about whether it was 'safe' to use their social media credentias to log in, especially since we're likely to be talking about new users (on the basis that your test was only run for a short period and that existing users are more likely to already be registered).

However, for whatever reason, your 'wish list' workflow/interface didn't trigger the same hesitancy. This could in fact be due to the fact there is a clear vertical divider on what is a newly routed page. Therefore, what you saw as a "pretty bad" interface was actually something that is working in your favour, especially if the higher registrations were not due to social media clicks.

In a nutshell, people have woken up to what really goes in with the big Facebook and Google enterprises in terms of datamining their personal details and web history and targeting them with 'personalised' advertising etc and more and more people are growing uncomfortable with this and are choosing to avoid using their social media sign ons for ecommerce sites.

You may be interested in the following article: http://www.techlicious.com/blog/should-you-use-facebook-or-google-to-log-in-to-other-sites/

I hope all this helps you pull something together for your presentation next week.

Answer 2

I may be completely off here, with my understanding of your question, and what you have tested and the outcome - I read through the comments to get as good an understanding as I could - but here are my thoughts:

Could the registration rate be down, because you are adding an extra step into process to create an account?

In your 'control', all the user has to do is to enter a password into the field and they can have their account created. However when you add the buttons, you are adding in an extra step, and what's more, is you are adding in a number of 'perceived' extra steps.

Before a user hits that button, they are wondering 'will I have to type in my fb/google username and password?', 'will I have to hit a button to approve this site?', 'do I want to give this site access to my fb/google information?' 'will I be taken away from this site?' - These are all, extra steps in the process, and extra steps are not great for when the user doesn't need to create an account to achieve what they want.

I say 'Before', because you might say, that the user doesn't actually have to do any extra steps, and particularly in this case where you are just testing the button and it doesn't actually do anything. When in fact the user is already thinking about how much effort a process might take, before they even start it. If they think it is too difficult or long, even incorrectly, some will drop off and not start it.

So it could be that the user didn't want to take too many steps, and seeing as they had already completed what they wanted to do, the need to register was so low that any extra steps didn't seem worth it.

Did you allow them to continue creating an account as normal? Because 'association' or the message that says you are only testing the feature could explain why even the normal registration went down during this time - if that was in fact the case.

Answer 3

The number of abandoned cart may not tell the full story. It's possible that the user is using a public/shared computer or mobile/desktop where they are not regularly logged into their social login provider, and when they see the social login button, they decided to abandon their cart so they can redo the transaction on their personal desktop/mobile, where they are logged in. This would have counted as abandoned cart in your logs, even though it is actually a successful social login. Moreover, I can see some users would then get annoyed when they figured out that the social login button doesn't actually work.

Another possibility is that the user may think that since you will be providing social login some time in the near future, they decided not to go through the hassle of creating an account now; and decided to wait until the social login is implemented instead to complete their shopping.

For a sample period of time

That is normally not how you should test things. For better comparison, you should have done A/B testing, to avoid periodic differences in the time period. This mean, show the social login buttons to half your users and not the other, and compare the performances between each halves.

There are too many factors to consider why the period associated with the social login button correlates with the effect that you are seeing.