7

I'm working on an application that's been in use for the past 3 years. It's a web-based product and we have been tracking its usage using Google analytics for the past 6 months (since the time I joined). Now I've been tasked to help them improve the product. I'm not sure where to start.

But one data point, that's glaring is - 90% of the users don't log into the application after the first time. The hypothesis is users don't find anything interesting to login again. Are there ways to probe this? I was thinking of usability study and probably sending out surveys. Request insights from your experience on how best to handle this? If usability study and surveys are the way to go - how do I focus on this?

Improve this question
4
  • 1
    Hey, Ben Brocka, why did you change the question? There was no "measure" aspect before...
    – Bartosz Rakowski
    Sep 12, 2012 at 13:25
  • @BartoszRakowski, the original title was way too broad and vague, so the question needed to be focused. If that's not the dimension along which Tara wanted to focus it, she can always edit.
    – Monica Cellio
    Sep 12, 2012 at 15:25
  • I've just remembered some presentation I saw previously: slideshare.net/mobile/msdnbelux/metro-design-deep-dive Slide #29
    – Bartosz Rakowski
    Sep 14, 2012 at 12:12
  • There are two major potential reasons: the application does not what the user expected it to do or it does not do so in a way the user likes. The first one would be very critical, since it means your application is useless for the people you attract. It's kind of hard to tell which is the reason by means of tracking, so a simple survey (or better: call it a reminder) for inactive users that presents "does not do what I want it to do", "too hard to use", "other" would not be the worst choice.
    – pmf
    Nov 8, 2017 at 13:53

5 Answers 5

Reset to default
3

90% of the users don't log into the application after the first time. The hypothesis is users don't find anything interesting to login again.

My hypothesis would be that it is related to the pattern in which users search for application online.

  1. Search for an application. Select the candidates to look in.
  2. Look in. Discard the ugly, untrusted, incomprehensible or with too high cost of entrance.
  3. Make the tests, fulfill simple goals, gather impressions. Compare.

So, your app may have problems with the last stage. You may now:

Check if the hypotesis is true. Send your users a "we miss you" email and ask for reasons. Ask real users with survey or do whatever you find applicable.

Invest in first time user's scenarios:

  • Find out the users goals, what actions or help to propose on a kick-out page etc.
  • Engage the user, show her how easy it is to use a system, make her believe it's usefull, easy to learn, don't leave her alone with the app on the initial stage.
  • Make it costly to leave (but with use of "white" UX patterns). For example, while teaching how your app works, show how easy it is to import or enter data into application. If you use elements of gamification - there would be a cost of abandoning the achievements or earned bonuses. Leaving the system would cost the learning time, time related to entering the data in another app, and so on...
  • Make it worthy to come back. It may be 50% rebate valid for next two weeks, it may be some related content which will be available soon for active users, a contest - in other words anything that is a "time offer".

Regardless of whether such hypothesis is true or not. Invest in studying the users thinking, needs and behaviour.

Improve this answer
2
  • Thanks there are some interesting points - I'll try out the 'we miss you' email and get see their response.
    – Tara
    Sep 13, 2012 at 14:00
  • 1
    It's worth to notice, 'we miss you' email is a channel of communication with all the related potential. It is also wise to select appropriate information and form in order to achieve a goal. Survey asking for reason should be one of the elements of such email. You can also offer a help if your users have problems with the site. Ask if they need a feature or forgot the password. Advertise what have you changed/added lately that is worth taking a look, etc.
    – Bartosz Rakowski
    Sep 13, 2012 at 19:19
2

The first two tools I'd be look at are, in order:

  • doing some quick usability tests to see how people are actually using the system

  • getting some input on what users are doing on-mass with the site via metrics/analytics

If the setup allowed it, I'd be trying to set both of these things up as regular processes (e.g. so that we were doing usability testing with a few users every couple of weeks) so that we could start getting a continual drip feed of feedback into the development process.

If the setup didn't allow that I'd be looking at ways to change that :-)

From your question it looks like you've already made a start at the metrics/analytics stuff. As to the underlying cause - I'd look to user testing first rather than surveys. Get some people in your target market that haven't registered in. Get them to register. Observe and reflect.

Improve this answer
2
  • Thanks Ardrian. That's interesting - is there a reason for running tests with the new users rather than the existing users?
    – Tara
    Sep 13, 2012 at 16:44
  • You said new users weren't coming back. If you test with users that are coming back you may not see the problem :-)
    – adrianh
    Sep 14, 2012 at 7:11
1

To be able to improve Usability, you need to measure the User Experience. As of now you have reports from Google Analytics, which is good. You know what the user does, how long she does it, but you still need to answer why. This could not be done by Google Analytics, you’ll need other tools.

To get a broad picture, a survey on the site/product is good to get a general picture of what several users think of your product. There are some free frameworks/sites to use for this purpose, such as Surveymonkey. With that result you’ll have input to the next step, such as an interview or usability test, with a few real user. Let them use your product and solve task that you know doesn’t work, and try to find out why it doesn’t. Ask your users and observe what they do. Sometimes users say one thing and do another.

When you have the result from the usability test, you have input to your UX team to improve the product. Make a few different interactive prototypes (or paper prototypes) and redo the usability test. Iterate until your users and yourself are satisfied with the outcome.

When you’re finished implementing the changes, do yet another usability test and see if the users approve of the new design. If so – launch and promote your product.

Tools reference: Measuring User Experience

Improve this answer
1
  • Thanks Benny. We have a 'Send us your feedback' link in the application and we are tagging it as well but people seem to be clicking on it and closing it - no responses through it - hence we dont have an answer to "why". I guess I'll send out surveys to solicit feedback on the application and then create paper prototypes.
    – Tara
    Sep 13, 2012 at 14:06
1

Email them and ask!

Specifically, automated lifecycle emails. Send an email to everyone who signs up but then doesn't use the app again. Ask for personal responses, and give a deal to everyone that responds. Extend their trial or give a discount off the first month.

It's awesome that you're tracking that 90% of users at all. Most of the work is already done.

Guide:
https://training.kalzumeus.com/newsletters/archive/lifecycle_emails_2

Thanks for signing up for the free trial of Appointment Reminder! We wanted to check in and see how things are going.

I run a small business myself, so I know things occasionally get busy. The computer says that you have scheduled less than a handful of appointments in Appointment Reminder so far. You've still got about 10 days left on your free trial, but we wanted to get in touch to see if we could help.

We've got two questions for you, if you'd care to answer:

1) Do you just need a bit more time? That is totally OK. We're happy to extend your trial by another month. Send me an email and I can set this up for you.

... etc...

Improve this answer
1
  • Thanks for your response. I sent out an email and then followed it with another reminder after about a week. I got no response though it was sent to around 100 customers. Before the follow up email, I tried calling a few customers but they were cold because they thought it was a sales call. Any tips to have a better success?
    – Tara
    Oct 4, 2012 at 18:20
0

A lack of relevancy.

The main reason why visitors would sign up and then leave. Is because they found nothing relevant for them. Either it was not relevant to a problem at hand, or it was not relevant to information they were seeking.

This can be caused by two possible reasons.

  1. The visitors arriving at the website are the wrong audience.
  2. The web application offers no meaningful benefit that the visitor finds relevant to their problem.

You would be amazed at how popular a horribly designed web app can be for people who find it very relevant, and at the same time cleanly polished professional apps can bomb if they lack relevance.

I know this wasn't the answer you were looking for, but it's the key as to why visitors leave and don't return.

Improve this answer
1
  • 1
    Thanks for sharing your thoughts. I agree that lack of relevance can be a potential reason. I'm looking ways to validate the hypothesis.
    – Tara
    Oct 4, 2012 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.