Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

I'm working on an application that's been in use for the past 3 years. It's a web-based product and we have been tracking its usage using Google analytics for the past 6 months (since the time I joined). Now I've been tasked to help them improve the product. I'm not sure where to start.

But one data point, that's glaring is - 90% of the users don't log into the application after the first time. The hypothesis is users don't find anything interesting to login again. Are there ways to probe this? I was thinking of usability study and probably sending out surveys. Request insights from your experience on how best to handle this? If usability study and surveys are the way to go - how do I focus on this?

share|improve this question
    
Hey, Ben Brocka, why did you change the question? There was no "measure" aspect before... –  Bartosz Rakowski Sep 12 '12 at 13:25
    
@BartoszRakowski, the original title was way too broad and vague, so the question needed to be focused. If that's not the dimension along which Tara wanted to focus it, she can always edit. –  Monica Cellio Sep 12 '12 at 15:25
    
I've just remembered some presentation I saw previously: slideshare.net/mobile/msdnbelux/metro-design-deep-dive Slide #29 –  Bartosz Rakowski Sep 14 '12 at 12:12
add comment

5 Answers

up vote 1 down vote accepted

90% of the users don't log into the application after the first time. The hypothesis is users don't find anything interesting to login again.

My hypothesis would be that it is related to the pattern in which users search for application online.

  1. Search for an application. Select the candidates to look in.
  2. Look in. Discard the ugly, untrusted, incomprehensible or with too high cost of entrance.
  3. Make the tests, fulfill simple goals, gather impressions. Compare.

So, your app may have problems with the last stage. You may now:

Check if the hypotesis is true. Send your users a "we miss you" email and ask for reasons. Ask real users with survey or do whatever you find applicable.

Invest in first time user's scenarios:

  • Find out the users goals, what actions or help to propose on a kick-out page etc.
  • Engage the user, show her how easy it is to use a system, make her believe it's usefull, easy to learn, don't leave her alone with the app on the initial stage.
  • Make it costly to leave (but with use of "white" UX patterns). For example, while teaching how your app works, show how easy it is to import or enter data into application. If you use elements of gamification - there would be a cost of abandoning the achievements or earned bonuses. Leaving the system would cost the learning time, time related to entering the data in another app, and so on...
  • Make it worthy to come back. It may be 50% rebate valid for next two weeks, it may be some related content which will be available soon for active users, a contest - in other words anything that is a "time offer".

Regardless of whether such hypothesis is true or not. Invest in studying the users thinking, needs and behaviour.

share|improve this answer
    
Thanks there are some interesting points - I'll try out the 'we miss you' email and get see their response. –  Tara Sep 13 '12 at 14:00
    
It's worth to notice, 'we miss you' email is a channel of communication with all the related potential. It is also wise to select appropriate information and form in order to achieve a goal. Survey asking for reason should be one of the elements of such email. You can also offer a help if your users have problems with the site. Ask if they need a feature or forgot the password. Advertise what have you changed/added lately that is worth taking a look, etc. –  Bartosz Rakowski Sep 13 '12 at 19:19
add comment

To be able to improve Usability, you need to measure the User Experience. As of now you have reports from Google Analytics, which is good. You know what the user does, how long she does it, but you still need to answer why. This could not be done by Google Analytics, you’ll need other tools.

To get a broad picture, a survey on the site/product is good to get a general picture of what several users think of your product. There are some free frameworks/sites to use for this purpose, such as Surveymonkey. With that result you’ll have input to the next step, such as an interview or usability test, with a few real user. Let them use your product and solve task that you know doesn’t work, and try to find out why it doesn’t. Ask your users and observe what they do. Sometimes users say one thing and do another.

When you have the result from the usability test, you have input to your UX team to improve the product. Make a few different interactive prototypes (or paper prototypes) and redo the usability test. Iterate until your users and yourself are satisfied with the outcome.

When you’re finished implementing the changes, do yet another usability test and see if the users approve of the new design. If so – launch and promote your product.


Tools reference: Measuring User Experience

share|improve this answer
    
Thanks Benny. We have a 'Send us your feedback' link in the application and we are tagging it as well but people seem to be clicking on it and closing it - no responses through it - hence we dont have an answer to "why". I guess I'll send out surveys to solicit feedback on the application and then create paper prototypes. –  Tara Sep 13 '12 at 14:06
add comment

The first two tools I'd be look at are, in order:

  • doing some quick usability tests to see how people are actually using the system

  • getting some input on what users are doing on-mass with the site via metrics/analytics

If the setup allowed it, I'd be trying to set both of these things up as regular processes (e.g. so that we were doing usability testing with a few users every couple of weeks) so that we could start getting a continual drip feed of feedback into the development process.

If the setup didn't allow that I'd be looking at ways to change that :-)

From your question it looks like you've already made a start at the metrics/analytics stuff. As to the underlying cause - I'd look to user testing first rather than surveys. Get some people in your target market that haven't registered in. Get them to register. Observe and reflect.

share|improve this answer
    
Thanks Ardrian. That's interesting - is there a reason for running tests with the new users rather than the existing users? –  Tara Sep 13 '12 at 16:44
    
You said new users weren't coming back. If you test with users that are coming back you may not see the problem :-) –  adrianh Sep 14 '12 at 7:11
add comment

Email them and ask!

Specifically, automated lifecycle emails. Send an email to everyone who signs up but then doesn't use the app again. Ask for personal responses, and give a deal to everyone that responds. Extend their trial or give a discount off the first month.

It's awesome that you're tracking that 90% of users at all. Most of the work is already done.

Guide:
https://training.kalzumeus.com/newsletters/archive/lifecycle_emails_2

Thanks for signing up for the free trial of Appointment Reminder! We wanted to check in and see how things are going.

I run a small business myself, so I know things occasionally get busy. The computer says that you have scheduled less than a handful of appointments in Appointment Reminder so far. You've still got about 10 days left on your free trial, but we wanted to get in touch to see if we could help.

We've got two questions for you, if you'd care to answer:

1) Do you just need a bit more time? That is totally OK. We're happy to extend your trial by another month. Send me an email and I can set this up for you.

... etc...

share|improve this answer
    
Thanks for your response. I sent out an email and then followed it with another reminder after about a week. I got no response though it was sent to around 100 customers. Before the follow up email, I tried calling a few customers but they were cold because they thought it was a sales call. Any tips to have a better success? –  Tara Oct 4 '12 at 18:20
add comment

A lack of relevancy.

The main reason why visitors would sign up and then leave. Is because they found nothing relevant for them. Either it was not relevant to a problem at hand, or it was not relevant to information they were seeking.

This can be caused by two possible reasons.

  1. The visitors arriving at the website are the wrong audience.
  2. The web application offers no meaningful benefit that the visitor finds relevant to their problem.

You would be amazed at how popular a horribly designed web app can be for people who find it very relevant, and at the same time cleanly polished professional apps can bomb if they lack relevance.

I know this wasn't the answer you were looking for, but it's the key as to why visitors leave and don't return.

share|improve this answer
1  
Thanks for sharing your thoughts. I agree that lack of relevance can be a potential reason. I'm looking ways to validate the hypothesis. –  Tara Oct 4 '12 at 18:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.