Mobile Augmented Reality App. What pattern for AR visualization and navigation?

Question

I am designing a mobile app with an AR functionality that allows users to detect a product label and get a bunch of information about it.

We want to display this content over the captured image. Content might be text and images and it could be quite long.

I have two questions:

1) to display content: is it better to use a 3D animation representing something like a folding notebook? Or using plane paginated tabs over the background image?

2) is it better using the real captured image as background, or substitute it with a static product image. This way people won't be forced to stay right in front of the label during the navigation.

These are the sketches I am doing:

Please add references if you find any.

Answer 1

1) to display content: is it better to use a 3D animation representing something like a folding notebook? Or using plane paginated tabs over the background image?

You don't provide any insight into the kind of information or it's structure. If this is just a question about the pro's and cons of a visual design... I'd say both options are viable, but it's usually easier to create a high class look and feel by keeping it plain. Unless you need a 3D animation to matches other parts of the design I'd keep away from such a thing. But that's almost personal preference.

2) is it better using the real captured image as background, or substitute it with a static product image. This way people won't be forced to stay right in front of the label during the navigation.

It could get really annoying if the information disappears as soon as the camera isn't looking at the product anymore. I'd propose capturing an image with the camera and keeping that as a static background behind the information. Alternatively you could use a stock image here, because now the image isn't the focus anymore but the information is. And only change the information that is displayed when a user actively scans a different product.

Answer 2

1) to display content: is it better to use a 3D animation representing something like a folding notebook? Or using plane paginated tabs over the background image?

Although the 3D animation might be really good eye candy, in my opinion at least, it gets real old real fast and is a sort of minimum 'lag' you're choosing to put in purposely. Meaning that until the animation completes the user won't see the information they're really trying to get at. This just gets annoying as time goes by. Just bring up the product info as fast as possible.

2) is it better using the real captured image as background, or substitute it with a static product image. This way people won't be forced to stay right in front of the label during the navigation.

Don't use the real captured image as the background unless you're going to start altering the color of your text based on the image background. Imagine the user's scans a primarily white boxed product. Now you have white text on white background making it hard to read. ( or a black box with black text ).

Instead, capture the image and allow the user to look at, switch to, the original picture if they want but when they read the info you provide it should be on a background that facilitates reading the info.

Also, as someone mentioned. Don't force the user to stand in front of the product. Allow them to snap a photo and fetch the info and look at it as they keep on walking. Much better experience than to have to stand in front of a something to read all about it.

Answer 3

Can you be a bit more specific about the kind of products you will be scanning using the app? I was imagining something in a supermarket scenario till I saw you talking about recipes in the image.

Anyways, moving on to your questions:

1. Is there any particular theme to your app (or the things which the users will be scanning)? Because, that will help in designing of the interface. Say you want to show just the detailed nutrition composition (more detailed than that on the box, ofcourse) of what product the user is holding, then it would just be a single page with the image and a list of the nutrients visualized in whatever manner you prefer. If it is much richer content, say you have related videos or images and other stuff, then those can be the tabs or sections in the book which the user can click to get more information.

2. The pros of using a stock image: if the stock image matches the product they are holding, then it reinforces the fact that the app is accurate. Also, when the user scans the product and the details are populated on the screen you can display a message saying that they no longer need to focus on the product and that it has been recorded. Since you are doing AR, I would assume you want the user to focus on the product while going through the details. For me that makes more sense to have an open space feel wherein I can point my phone anywhere and whatever product is in focus has its details available at my fingertips. If this does not fit your use case then you can go for the above mentioned stock image solution.