I have an interesting set of constraints to work with:

-We're launching a new line of products early next year. In a few weeks, we'll be sending out beta software/hardware units for distributors to evaluate. None of them are geographically accessible.

-There are no dedicated UX employees. I'm the de facto usability guy due to my human factor background, but it's not my primary role within the company.

So I'm short on time, access, and money. What exactly can I do to gather high quality data? Right now I'm thinking that I could send out surveys, test it on employees unfamiliar with the product, or recruit friends on my time off.

3 Answers 3


Test it rather than send out surveys

If you are involved with it you'll already know how it should work so get some fresh eyes on it and watch them like a hawk.

Testing it on 5 people is a lot better than not testing it at all


Was there any prototyping or usability testing done locally before now and can you contact/get access to those people?

(Ideally, this should be an ongoing part of the development process over time & you'll have a list of interested early adopters. If not this time then for the next one! :)).

If not, can you get to anybody at all who fits the target demographic (& can legally see the thing)?

Try to get "out of the office" if at all possible - take it to a local [place where target users are] and let people play with it (if doing so won't get you fired). If you can't take the real thing, maybe a mockup or stand-in that won't give away the game but can be used to answer the same questions?

Surveys are ok, but again you need an audience. Although self-selecting at least it's free feedback that's easy collect/analyze online. Can be hard to write good questions though & mouse/pencil-whipping is a risk.

Employee testing can be better than nothing, if they're not familiar w/the product & don't have habits or knowledge a typical user wouldn't. Don't want to reinforce biases or assumptions, but if that's really all you have go for it.

Last thought: Do you have access to any potential users in the distributor's areas? Would be a good opportunity to gauge local feedback/demand as well (again, probably should've been part of the process before now). Are the distributors going to do any usability testing on their end w/data you could get access to?

Use Skype/Google+ (or a phone call? Crazy talk...) to schedule & conduct sessions at a distance, screen capture if possible with the user talking out loud to at least hear them say what's confusing, broken, or awesome. If not real time, ask them to record short videos. Similar to what you'd do in person, just without the same opportunity for immediate follow-up.

  • Thanks for the well thought out response. AFAIK, they haven't done any kind of continuous usability testing. They've done analytics here any there, but that's about it. They're going to do an internal alpha at some point, so I'll at least have a chance to working with a captive audience.
    – dmacfour
    Commented Oct 20, 2014 at 23:55

First of all don't panic! ;) You need staying power.

Even if you feel in charge (which you aren't by role) - your company isn't ripe for usability testing yet. So, you need to show them the benefits first. It is a long path for you to build up awareness for users and to establish design processes. Look out for UX maturity docs - how to reach it strategically and what to do tactically.

Survey's are great for you, because you just set it up and it's running. It shows numbers as results, which I think is important for your target audience - marketing or product management. But don't let distributors fill them out. Sales people want to sell and would tell you a broken car is a great deal. Their feedback isn't honest in this case. Think about how you get real customers.

Usability tests with employees is good. You will find the biggest problems and some minor problems. Even if they are no real users and do real tasks, its better than nothing.

But be aware! It doesn't make sense, if you haven't the time to analyse, document and present the outcome. And you will need development ressources for implement the findings. Otherwise you will end up saying in half a year "Oh well, I've seen this errors and feedback some months ago during my usability test. I told you twice. But nobody heard at me - you were busy with bug fixing. Well next time, you better listen to me." You won't end up like an egghead, won't you? Personally I had this situation last project (without the egghead sentence).

See - first you need awareness and credit for user testing! Take the low hanging fruits first. Look who to convince, how she/he thinks and choose the methods upon it. I did change management twice and it needs some time. Sadly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.