Hot answers tagged

53

The fewer words the better, and no words at all are better than negative words. Don't say why you think there might be a problem, or even that you think there is likely to be a problem. Instead just make it easy for them to contact you in the event that they do happen to come across a problem. I quite liked an experience I had recently at surfdome where it ...


5

The only thing you need to do is to collect 3-5 users and conduct a simple user test. Preferably some of the users that would actually be among your end users - but that's not that important right now. Ask them, one by one, to carry out a certain task on your app. Observe! Don't tell. When they struggle: Ask why. When they succeed: Ask why. I can assure ...


3

I think it really depends on the release cycles of product updates and whether the releases are incremental or if a lot of changes are introduced at the same time. Of course, people are now used to the idea of Google release updates constantly (and some just automate the process to apply the updates as well), but this obviously doesn't apply to every ...


3

UAT is typically used for functional acceptance testing in waterfall approach. That is, a business analyst has taken the user input in coming up with the specs, or a set of feature requirements at the beginning of the project. The users will now verify that the system behaves as expected based on the feature requirements. Questions like: Does the system do ...


2

Personally I would favour both. As you say the beta channel approach can lead to issues with power-users not doing things standard users might, in the simplest example a power user might know not to put text in a field asking for a number so you may miss the errors that would indicate you forgot to validate the entry until it goes live and lots of non-...


2

For me it sounds like your are doing customer development rather than user treting, because you need feedback about the product value and its actual use. Anyway the methods of custdev are similar to user research, not before concept phase like research but after beta phase. We just ended our beta phase and did this: . focus groups with some users . ...


2

This is an awesome opportunity! I've often used beta programs as part of my UX research strategy. Using your beta program to get directed UX feedback can be an excellent way to get highly-actionable feedback. How to structure it depends on your timeframe, needs, number of users, and resources. Some options include: An online forum where you engage ...


1

I usually kick off with around 12-15 users. In terms of what questions to ask really depends on the type of site you are building. For example, user interview questions for an eLearning app could include the following: How they learn a new concept online. What steps do they follow? Describe a typical day at school/work. What makes a good day in their ...


1

There is not a certain number of testers. You must determine the goals of your analysis. Also you must determine the planned number of users of your product. After that you can understand what types of testers you need, how many types and how many testers you need.


1

Why are you talking hard numbers ? The whole point of the Beta program is to get users to test a new version, you want them to test the program extensively and report any issues. Ideally, you would want the techier tier of your user base, because they have more experience in finding bugs, that are also fanboys, as fanboys are eager to help squash any bugs ...


1

It depends on what kind of research questions you have and what kind of data you want to collect. The Qualtrics tool is great for any kind of quantitative analysis. Think of the margin of error as a kind of sensitivity tool: if there is an issue that affects 5% of users or less, you won't be able to detect it for sure if you have a margin of error of 5%. ...


1

Successes and failures are specific to your app. You should define metrics that aligned on your business objectives / UX design choices. Examples of metrics can be objective measures such as error rates, completion time, time spent on app, or subjective such as "how easy - difficult would you rate this app", etc. If you're looking for more qualitative and ...


1

Edit: As I haven't made any advise on any product to measure A/B testing at it's best, I assumed you also might find a tool by googling as this sample research gives a couple of examples. I'm using my company's own product to measure the A/B test on e-mails but at the end of the day, I prefer to take the data from there and use them under one roof I created ...


1

For your app to succeed it needs to work perfectly, otherwise it will just attract bad reviews. Think about what the big guys do: new features are listed as pending in a future release, and only added when they work.


1

Technically speaking, at no time, because usability testing comes BEFORE beta. However, if you didn't do usability testing before this stage, then I'd recommend you to do it right at the beginning to avoid contamination and confound variables such as Hawthorne effect. This is specially true if your product's features are somehow special or representing new ...


1

The best experience I've had for managing expectations in beta phase was with Adobe Xd and their online forum / support. You can not only check if features have already been requested by other users, but you can also vote for the ones you really want. When they released a feature I had voted for, I received a specific email informing me of the new feature. ...


1

The main purpose of user acceptance testing is determining whether the application under test is fit-for-use by the business user (with a focus on functionality). It is good practice to involve end users in this stage of testing. Test strategy and test design are performed by QA. A typical testing process in a large organisation (waterfall) goes through ...


1

I would do two things: The first would be to carry out an observation with around 8 users, asking them to "think aloud" throughout. This will give you a good idea of what users might think when they start using your app. If it looks like the protocol is distracting the user, tell them to only comment when there is something they don't like/understand. ...


Only top voted, non community-wiki answers of a minimum length are eligible