Fallback for MultiTouch pan and zoom

Question

A planned app that will run on tablets that support multitouch needs a graceful fallback for tablets and desktops which don't. I have a view onto a possibly very large 2D electron microscopy image which can be panned and zoomed. This is what I've come up with:

In more detail: Blobs are translucent and represent the finger positions. Dragging a blob moves the point directly under it. That way you can both pan and zoom. Blobs are added by clicking, but on adding a third blob you are back to no blobs. Blobs are 'in use' if the mouse is over one and the mouse button is down (and possibly moving and dragging that blob). If not used for 2 seconds, the blobs fade out, taking 0.5 seconds to do so. If you click somewhere else so that the blobs lose focus, then they vanish immediately.

The same zoom and pan method will also be used for a timeline, where naturally it will only zoom and pan in the one dimension.

I'd like suggestions that will help me get the details right.

• Is the fading out on no-use good, or should the blobs stay until some user action makes them go?
• On attempting to add a third blob should I go back to no blobs, or back to one?
• Should I for consistency show these blobs on multitouch tablets, even though they are not needed there?

I feel the rough design shows promise. I feel tweaks to it could turn it from a design that is workable, but seems a bit odd, to one that is lovely to use. I'm therefore not just interested in answers to the three specific questions, but interested in any suggestions that will make it easier, more pleasant, more intuitive to use.

---

EDIT 1

One specific difficulty is what to do if there are quantized values, e.g. if we only allow rotations by multiples of 15 degrees, or even not at all, or if we want to use zoom factors that are whole numbers. In those cases the blobs can't be pinned exactly to the image, and it becomes less intuitive what is going on.

I'm also concerned about discoverability. Zooming often adds + or - graphics to the cursor.

I reckon I'm going to need different modes, one mode for panning-zooming-rotating, and another mode for lasso selection. I can put a tootltip on the mode selector button which will help discoverability.

Answers so far are not helping me progress this as much as I would like, so I've held off on selecting a best answer so far.

---

EDIT 2

Some responders are thinking that the multitouch pan/zoom is just gestures, in the sense that a single gesture is recognized and once recognized triggers a fixed action. On the tablet the points touched follow the fingers - at least they do whilst you're touching the tablet. Pan, zoom and rotate are ballistic, i.e. movement keeps some momentum on finger up, but only if above a set speed - so it does not happen for slow/precise movements. Users tend to keep the left-mouse-button down after a quick move, and that is equivalent to not taking finger off screen, so whilst I'll have ballistic behavior for mouse use I don't think it's going to be used as much on desktops as it is on tablets.

Answer 1

1. The fading out is good, I think. Otherwise you would need a way to get rid of the blobs.

2. I feel that going back to no blobs on the third is confusing and unclear. Also, it takes another two clicks to get back to two blobs. On adding a third blob I would remove the first one, leaving you with two blobs - 2nd and 3rd. On adding a fourth we remove the 2nd and so on. If the user doesn't like them, they fade out in any case. It makes sense for the clicks to always have the same effect, namely adding a blob, rather than adding a blob in some cases, and clearing all in other cases. I think that the concept of FIFO is easier to grasp than the concept of reversing the role of the click.

3. No. Many apps have slightly different versions for no-touch, single-touch and multi-touch interfaces, and that's fine. Adapting an app to each device is common practice. On a multi-touch device it would just send the wrong message, and users might think that they need to use the blobs.

EDIT: James, concerning your update: I agree that rotating by fixed angles makes it less intuitive, but I think you can handle it by making these blobs leave a small trail before the switching point. Let’s assume that you rotate the image by 15 degrees. The user touches the blobs and begins rotating it - and there's a visual indication that this is a known and acceptable behavior - the blob responds and moves, leaving a transparent shadow trail behind it - until the user gets to the 15 degree point, at which moment the image rotates as well, and the trail disappears. If the user lets go of the blob before getting to 15 degrees, the blob bounces back to its original position. Also, it would be best to mark the 15 degrees positions on the screen as soon as the user begins rotating - you can make them look like possible positions/placeholders for the blobs, suggesting to the user that the blobs must fall into place at those locations. And I agree concerning the two modes.

EDIT 2: Concerning the visualization of blob markers - in single-touch, you only have one active blob, and the other is the anchor. So you need to show only the markers for the possible positions of the active one (and switch if the selection changes). This is a crude drawing of what I had in mind. And the arrows coming out of the blob would help too. Of course, the markers only make sense when there aren’t an awful lot of them. Much depends on the graphics, as usual.

Answer 2

Good question! The true solution will come from usability studies conducted on the device you are working on. However, generic best-known methods would indicate something in the following lines:

1. Controls should not fade out. That would be the system taking over decisions (the decision to operate or not) from the user.

2. You should go back to no blobs. This fulfills the double intention of removing blobs from the UI -to enhance visibility, for example- and leaving the decision of when to remove controls from the UI to the user.

3. If the touch-gestures you are using are standard to the device, then you can do without the blobs in the multitouch interface.

Answer 3

On the CHI2010 a group of researchers suggested a gesture named "Cyclo star", which consists of a circle gesture to initiate zooming. After the initiation you can just carry on "drawing circles" to zoom in, similar to operating a wheel.

You can see it in this video its described and evaluated in a this paper: http:// dmrussell.net/CHI2010/docs/p2615.pdf

I agree as well with what Nikita Prokopov pointed out: Widgets do not hurt. See Don Normans Essay on NUIs for a discussion.

Answer 4

I think you should go with simplest and traditional solution: just add toolbox to the corner and allow content dragging anywhere.

1. It's easy to discover them and they are familiar to most users.
2. You can do quantized values easy.
3. You can make it semitransparent if you need space.

I also think that mouse users do not really concern if you have multitouch support or not and how it works, so don’t be afraid creating different controls for multi-touch and non-multitouch users.

Example follows:

Answer 5

How to enable pan, zoom and rotation with no control points on screen.

• To Pan - simple drag.
  
  

• To Zoom/Rotate:
  

  • With touch but no multi-touch:
    

    • Long press and then drag (without releasing) up/down in for zoom in/out.
    • Long press and then drag (without releasing) around for rotate roll (around viewer).
    • Long press and then drag (without releasing) sideways for rotate heading (sideways).
    • Long press and then drag (without releasing) up/down WITH large arc to either side for rotate pitch (up/down).
      
      

  • Desktop (no touch):
    

    • Ctrl+Scroll for zoom in/out.
    • Shift+Scroll for rotate roll.
    • Shift+drag left/right for rotate heading.
    • Shift+drag up/down for rotate pitch.

Answer 6

I'm not understanding the need for this blob idea and it looks very unintuitive. You haven't provided enough motivation about why you can't use standard UI approaches for this. You're trying to simulate a multitouch experience on a non-multitouch device which is just the wrong way to go about things in my opinion.

Look at first person shooters on consoles and PCs: on consoles the gamepad and analog sticks are used for looking and moving and on the PC a combination of the mouse and keyboard is used. These control schemes are specifically designed to take advantage of the strengths of the input devices available. What you're doing to something like trying to simulate a mouse and keyboard using a gamepad when you really need to be thinking how the gamepad should be used to do the task at hand.

I don't see anything wrong with using the tried and testing "pan tool" and "zoom tool" approach or having e.g. hold down the left mouse button and drag to pan and use the right mouse button for zoom.

Answer 7

I am not very clear as to the need of these blobs. The concept seems a bit confusing. Here's my understanding:

How many tablets don't support multitouch? If only 1 or 2 you could ignore them - but you'd need a fall back for desktops since they are a point and click interface. You could simply go with the google maps approach - scroll for zoom, click and drag for panning. For the third option (rotate about a point) the most intuitive one in my opinion, with a mouse, is click and scroll (i.e., have the left mouse button pressed and scrolling with the other finger - or have the right button pressed with the scroll, whatever is easier for the user).

This may not be directly related, but have you read "The design of everyday things" by Donald Norman? It uses real world examples (non-software e.g., doors, shoe laces etc) and highlights the important considerations as what to hide when and when is it a bad idea or how does the layout affect the mental perception etc.,. Pablo's points also seem to reflect the same wisdom. The book is an entertaining read and packed with tips for UI design.

Hope that helps.