3

The situation is this. My company is designing a native app for Android and iOS simultaneously. The timeline is very tight but we are going to squeeze in three rounds of user testing.

With such a short timeline, I would like to avoid having to prototype, screen, recruit, create test plans, and coordinate participants for both iOS and for Android if at all possible.

Will this be a big miss for me if I focus on one of the platforms and use those findings to inform the MVP for both? The design should be pretty straight forward, not using many uncommon interactions.

2

In your situation it is justified to test on one platform.

iOS and Android generally only differ in OS level functionality, so once the user enters the app it's essentially the same user experience.


The areas where it may differ are:

  • Swiping from the edge of the screen might reveal OS overlay
    • i.e. Notifications overlay.
  • Access to OS level functions
    • Camera, Photos, Email, etc.
  • What happens when a notification appears?
    • Will the notification banner conceal important information?

But... informal testing is better than no testing at all

Test it informally on both platforms with colleagues/family/friends if you can.

In my experience, an informal test will discover any significant usability issues regarding OS differences on mobile.

Alan Cooper promotes the use of informal testing sessions where formal usability tests are not viable:

The informal style can be done spontaneously and requires less preparation.

It can be a useful alternative to usability testing when the design team doesn't have time to prepare for formal usability testing.

0

Will there be (significant) differences between both versions?

Yes? Then you should test both. Because different designs obviously lead to different experiences, which should be tested individually.

No? Then you should combine the tests in one event. Most development tools allow exporting to multiple platforms, making the overhead for extra builds very low.

And for the test themselves you only need to add a checkbox on top of the feedback page to differentiate between ios/android versions. Because (at least theoretically) the apps are the same, the questions will be the same.

But this does have the benefit of spotting differences and potentially big issues on either platform. For example [cancel][continue] vs [continue][cancel].

In fact, this could even save you some time/effort, because you don't have to reject would-be testers for not using a certain platform.

0
  1. Consider iOS users for iOS tests and same with android.
  2. You should test both platforms. But you can start with one (f.ex. more popular in target group) — test it twice to nail your specific app functionalities and communication, than design for second platform and test it if possible.
  3. The most important are your specific app functionalities, communication, patterns from os shouldn't influence overall experience that much.
0

Yes, yes, yes, yes, yes

You must test on both platforms, otherwise you're leaving the door open to lots of conflicts. You say both platforms share a lot in common. Thus, IN THEORY should be the same for users. But... can you guarantee the coding on both platforms is flawless, that there are no mistakes and that potential platform issues have been covered? Can you guarantee it 100%? Without a doubt? If not... test!

As for times being tight, this is pretty common. Having time is a luxury most of the times we don't have. Blame it to bad planning, unrealistic expectations, further discoveries, cross-platform issues... well, it's pretty common. But there are not excuses for at least going to a college or mall and at least do some quick informal testing, just like Joel Tebbet said. No testing is way worse than poor testing.

The kind of problems you may have could be really weird. Coding issues, interaction issues, transition times, platform based expectations and many more.

If you want an example of what could happen, here's one big failure from yours truly:

we designed and started to built a great app, something we were really proud of. When testing the main feature, it worked like a charm. iPhone5 and 6, Galaxy5, Xperia... simply perfect. You could say testing shown our work was flawless! Then we tested on a low end Motorola: we burnt it. Literally: it started to heat and catch fire.

We tried to find out the issue until we realized the computational needs exceeded almost any low level smartphone. As simple as that. Just imagine if we launched something tested on Android and iOS only that high end phones. Happily, we discovered the problem at that stage.

Point is: not all issues and problems are those that you may reasonably expect. Testing is a tool to show you what you're missing (hence why you test with users and not stakeholders)

In short

There are no excuses to avoid testing. In any case, you won't be able to do the perfect test you want to do, but something is better than nothing.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.