1

I am currently the sole UX designer on my product team and have been tasked with coming up with a strategy and schedule for our upcoming beta programs for testing a new hardware product and application redesign.

Has anyone else experienced this intersection of usability testing and beta testing? If so, what is your advice for effectively aligning the UX and beta efforts for optimal testing results?

Improve this question
2
  • 1
    Do you have control over the beta product and its users, or is the product just being released to the masses in "beta form"?
    – Nicholas Pappas
    Mar 15, 2016 at 19:43
  • The product is a hardware device that syncs to iOS and Android applications. One beta will focus on an app redesign, and one will focus on testing the experience of a new hardware piece in addition to the newly designed UX + UI of the app. Part of my task is to geo-target our user base and recruit accordingly for a limited amount of devices that are available for our testing.
    – Malia Eugenio
    Mar 17, 2016 at 17:47

3 Answers 3

Reset to default
2

This is an awesome opportunity! I've often used beta programs as part of my UX research strategy. Using your beta program to get directed UX feedback can be an excellent way to get highly-actionable feedback. How to structure it depends on your timeframe, needs, number of users, and resources.

Some options include:

  • An online forum where you engage with your beta participants. Let them start threads (and respond to them as appropriate), and start your own as well. Depending on the number of users and their level of engagement, I limit my new threads to no more than one per day. Some example topics include asking them to complete a specific task (first-time setup, registration, ... ) or share feedback about a given feature. Mix up your questions to keep it interesting and encourage engagement across a broad mix of people. This is a great time to get feedback about an important workflow or feature, or perhaps that one thing that you never felt you got quite right, or that workflow that you had to deliver 80% of instead of 100% because the schedule got too tight.
  • Send a Smile! In short, the user takes a screenshot and give it a smile or a frown, and (optionally) comments on what is making them smile/frown. There's a longer discussion of this on the Office 2010 engineering blog.
  • Conduct a survey with beta participants. If you have a lengthy beta period, or if you gradually increase the number of participants in your beta, you can conduct multiple surveys.
  • If you conduct multiple surveys, you can track participants' responses over time. I especially like to gather satisfaction data on first use, on completion of the beta program, and at least once between the two.
  • Use your beta program to recruit participants for a standard usability study.

Always keep in mind that beta participants are rarely representative of your typical users, and are generally both more engaged and more technically savvy. Supplement the UX feedback that you gather from beta participants with other research that is more representative, if possible.

Improve this answer
1
  • Thanks so much, Nadyne! This is super helpful information and great points that I will definitely keep in mind as I draft my proposed schedule for user testing and beta. I appreciate your time and sharing these ideas!
    – Malia Eugenio
    Mar 17, 2016 at 18:10
0

Beta testing falls within the realm of UX testing - does the application/product/whatever work properly and is it easy to understand and use? Beta testing is merely a specific time-frame within the development process, whereas UX testing should be involved throughout the entire process.

Improve this answer
1
  • Seeing beta as a component of testing that falls into UX testing in a specific time frame makes a lot of sense to me. Much of what I am trying to achieve is finding a way to align UX testing and beta in a way that they can inform each other and lead to the most optimal insights so we can iterate quicker. I guess, in the grand scheme of things, it really all comes down to timing.
    – Malia Eugenio
    Mar 17, 2016 at 18:13
0

Usability testing can (and typically does) take place at any stage of the software development lifecycle, so beta testing is just a point in time when you decide to go and do the testing part of your overall UX design process. The key point is to try and collect information that will help you with future UX research and testing activities, taking into account the potential pitfalls of beta testing and trying to complement it with other forms of testing.

Personally I don't see UX and beta testing as different activities, it is more of a continuous process in which testing at a particular point of the software release is seen as doing beta testing. Of course, this is purely from a UX perspective as beta testing has other implications in terms of analyzing software performance, product development pipeline and marketing activities as well, so it is very useful to communicate with other teams to make sure you don't miss out on vital pieces of information that will help improve the overall user experience.

Improve this answer
1
  • Thanks for sharing, Michael! These are really great insights and you bring up a very good point - UX and beta testing are certainly one in the same and both serve the same purpose in the product development process. Cheers!
    – Malia Eugenio
    Mar 17, 2016 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.