3

I'm working on improving a product that rarely gets new customers (since it targets a very small specialized niche), and I'm not sure how usability testing would work here. I assume that current clients are already used to the issues of the current product, and would struggle with the changes I'd make (even if they were improvements) since it's changing what they're used to.

So my question is: how do I test with current clients, and use that data to measure improvements?

Improve this question

2 Answers 2

Reset to default
6

There's always going to be resistance to change, but this actually works out pretty well in the end.

Look at it this way, whether you test with existing users or new users, your existing userbase (the vast majority of your users) are still going to have to adapt to the new interface. So that has to be taken into consideration and kept in mind during your redesign.

Even if completely changing the UI metaphor would produce a more intuitive and elegant design, you still have to weigh the benefits against the cost of making all of your current users learn a completely new interface/workflow.

However, if your changes produce a significant enough improvement (relative to the learning curve) in terms of productivity / ease of use / time savings, then they'll still prefer the changes over the old design. The only people who won't be willing to make that concession are going to be the more technologically-illterate/technophobic users, and hopefully that's only a small percentage of your users. If not, then you'll have to decide:

  • What's the risk of alienating my current userbase and driving them away? Have our core userbase stuck with us over the years because we've kept a consistent interface that they're very familiar and comfortable with?
  • Do we want to ditch our current userbase and instead target a brand new user segment (maybe even one that's younger, more tech-savvy, more adaptable)? Or do we want to create a new product to service both segments?

Alternatively, you could do what Google has been doing and slowly phase people into the new interface. Give them a chance to preview the new interface and test it for you, and then after you deploy it, still allow users to continue using the older interface until a majority of users have switched over.

If you just want to measure whether your changes are better or not, and not have the data skewed by your test users' reluctance to change, then I suppose you'll have to test over a longer period of time to provide those users with an adjustment period. Run a series of 5-10 tests over 3-4 weeks and see if the users prefer the changes over the existing interface by the last test.

Improve this answer
1
  • Great answer! thanks. What you haven't covered though is how I would "detect" issues when they're actually used to them. I want to use those data to count the number of issues before and after the change. I'm guessing i'll have to closely watch their behavior and try to figure out their struggle points.
    – Mashhoor
    Dec 18, 2011 at 13:57
0

It depends on what access you have to the users. Generally if you don't get new users, then customer support doesn't give you as much information to work with. Also, if the software is not frequently updated then it can be hard to build up incremental knowledge. For a desktop application it is definitely more difficult to introduce subtle changes and test their effect, whereas web applications can be tested more easily. In your situation (and without more detail) the best way is to either provide ways for them to send direct feedback (if the customer base is not too large) or send out some survey and set a benchmark that you can measure again periodically and track.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.