User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It only takes a minute to sign up.
Sign up to join this community
Being very new to this field, I thought experts here could show me how to test this.
We have three different copies of an appstore description and we would like to see which one leads to more downloading. They are similar to each other, not drastically different.
I thought of usability hub, but the cost limits the number of users we can test.
So, what is the best way to hold this testing with a high number of people?
So does each of these descriptions has anything that distinguish it from the others? For example, does the 1st one talks more about "profitability" ("you can earn money/knowledge/friends if you download this!"), while the 2nd is more about "others are using it, so you should too" ("90% of Nsadaq 100s' companies are using this app, so should you")? If so, this is a matter of voice & tone that your product should reflect through all of its interactions with the user. The UI micro-copy cannot differ drastically from what you say in the app-store description.
If you're not sure about the type of voice & tone of the app, hence you've yet to decide what kind of description should fit your preffered audience, i would try to A/B test. Here are some links to companies that do such thing:
The reason i wouldn't go with user testing here, is that you would hear few people saying what they feel like when reading your descriptions. it's far from valid, even qualitatively-speaking. It's OK to do if you have some personas that you already defined, but if it's just random users - i would avoid it. it will mostly add many question marks into your team's discussion...
Good luck
Do you have your own website somewhere for the app? You could add a randomizer that displays (and tallies!) one of three versions, and then see which generates the most on-site downloads and/or passthroughs to the appstore.
we would like to see which one leads to more downloading
First things first - any experiments you run are synthetic models. You can certainly try to predict how some people will behave under simulated conditions with A/B experiments like this, but you have to know that even if your experiment is internally valid and sound a "winner" emerges, it doesn't mean it's externally valid and can be extrapolated to predict the behavior of App Store users in the wild.
Unless you're actually testing a few variants live in the App Store, you have to take the results with a grain of salt because they're just a model, and models are really only as strong and reliable as the amount of time and money you're willing to cough up to make it more of a realistic simulation...in which case you just as well do the real thing.
You mentioned cost is an issue, so it's important that you keep in mind the limits of the method because confidence costs $.
That aside, your question of cause-and-effect for a single independent variable lends itself perfectly to the format of A/B testing.
Your particular use case is closer to marketing conversion testing, a niche subset of testing which has its own hordes of expert tools and communities that include Optimizely, Unbounce, even Hubspot.
