12

If you want to make a usability expert twitchy and uncomfortable, ask her how companies like Apple can create acclaimed, wildly successful, and usable products using top-down design without usability testing. For extra discomfort, follow up by asking why there's no evidence that user-centered design works.

-- Top challenges to the practice of usabilty, Joshua Ledwell

(According to Spool, in the last 30 years, there has not been one website or other digital innovation that can point back to usability testing as the defining factor for its success.)

Can you name an actual company/product that has a really fantastic UI that got there by rigorous usability testing? (Please include only one company/product per answer.)

Improve this question
13
  • 5
    Apple products are not usable - Apple are just good at hiding the usability problems (example: I have an iPhone - the on-screen keyboard in great for typing but totally useless for editing because the only way to position the cursor is to achieve sub-pixel precision with my big fat finger - the result is that writing long e-mails is painful - not what I would call usable, unless I act like Steve Jobs and answer all my customers with rude one line messages)
    – Nir
    Commented Oct 27, 2010 at 10:11
  • 1
    I'm adding this because I'm sure otherwise all the Apple fans in the world would answer my previous comment telling me about the magnifying glass thingy that pops up if you leave your finger on the screen - I know about it, I use it, the cursor always moves a little when I'm pulling my finger off the screen after positioning it with the magnifying thingy - another example of a beautiful and very cool looking UI element with severe usability problems everybody ignores because it's so beautiful and cool.
    – Nir
    Commented Oct 27, 2010 at 10:16
  • 1
    @Nir: I think around the time of the original Macintosh, Apple was the king of usability. Now, they are merely good at it, which means they sometimes don't get things right--or it takes a version or three to get them right. They make beautiful products that people really like, though, and they're really on a roll lately in terms of innovation and marketing. The particular problem of positioning a cursor on a touch screen is a tough nut to crack. On my touchscreen phone (not an iPhone), I just stab repeatedly until the cursor goes where I want it (i.e., brute force approach) :)
    – devuxer
    Commented Oct 27, 2010 at 18:04
  • 1
    @nir...have you used ANY handheld device that has amazingly usable text entry capabilities? (I haven't...I kind of figure anything smaller than my hand isn't ever really going to be the most usable way to get type onto a screen)
    – DA01
    Commented Oct 28, 2010 at 1:10
  • 1
    @Nir It's easy to say "Apple just have to do X and everything would be way more usable" but I find your idea questionable and certainly in the face of not seeing better solutions from either Apple or Google, find it hard to imagine that there's some kind of magic cure-all lying around waiting to happen that no one's implementing because it "would look awful in demos". Sorry dude. Not buying it. Just like I'm not buying your assertion that Apple makes "not usable" products (millions use them just fine) or that people only buy them "because they're beautiful".
    – Rahul
    Commented Oct 28, 2010 at 11:19

7 Answers 7

Reset to default
18

My answer would be: usability testing isn't intended to design anything, it's intended to efficiently locate problems with a design. Good design requires vision, one or more talented designers, brainstorming, thinking things through, knowledge about your intended users, good design and prototyping tools, etc.

One thing surprises me about the quote, though. It implies that Apple doesn't religiously usability test their products. Is that really true? Note: usability testing is not the same thing as user-centered design. It's possible to design something without the participation of potential users, yet still test a design with users once you've created something.

Improve this answer
5
  • 2
    +1 for the same reason that software isn't designed by troubleshooting (but that's no reason to say you can't design with troubleshooting in mind - i.e. by using specific, context-appropriate error messages)
    – danlefree
    Commented Oct 27, 2010 at 4:01
  • 1
    At the Mac OS X Lion keynote, Steve Jobs specifically mentioned having user tested capacitive multi-touch monitors and talked about how it doesn't work well. So yes, they test with users. The rumor/quote about Apple not "testing" refers more to what you're saying about having initial vision and great design - they don't design by committee, they just have great designers.
    – Rahul
    Commented Oct 27, 2010 at 8:27
  • @Rahul: Thanks for confirming my suspicion :) You can really tell Apple designs are visionary and very not "by committee", but at the same time, they work well enough out of the box that you have to assume they've been put in front of plenty of users before releasing to the public.
    – devuxer
    Commented Oct 27, 2010 at 17:52
  • 1
    I think the famous quote is that Jobs doesn't do focus groups...I've never heard him say they don't user test.
    – DA01
    Commented Oct 28, 2010 at 1:13
  • "Usability testing" is a broad term, but one thing Apple definitely never does is recruit non-employees to come in and sit down at a computer to evaluate software. Note that the vast majority of Apple's high-value software is developed in extreme secrecy. nytimes.com/2009/06/23/technology/23apple.html
    – Dan Fabulich
    Commented Oct 28, 2010 at 3:10
6

The purpose of testing is to validate designs...not create them.

Improve this answer
3

Rather than listing a company or product (I'm not aware of one), let me suggest why this might be so.

My belief is that good design starts with a good design as a base. User testing only refines and optimizes the base and not all bases are created equal. If you start with a bad design base, user testing won't help you reach the optimal great design overall, it will only help you make that particular base the best it can be. If your base is fatally flawed from the beginning, your product will also be.

In addition to the base design, I think one must look at the overall production strategy. A UI isn't just the design, it's the philosophy behind the design. It's how the design functions and feels. Apple is one of the very few companies that controls their entire production line and the customer experience (through their stores, app stores, and iTunes). They make a concerted effort to make everything related to their products and user experience as seamless as possible and try to retain as much control as they can. One company in charge of a UI/UX will usually be more polished than multiple companies trying to do the same thing together simply because of communication issues, varied priorities, and different design philosophies and implementations.

Does Apple get everything right? Not at all, but their integrated design philosophy helps give them a competitive advantage. However, I'd venture to say that their great initial design could be improved even further with user testing.

Improve this answer
1
  • +1 for "User testing only refines and optimizes the base and not all bases are created equal"
    – James Crook
    Commented May 28, 2011 at 16:39
2

alt text

Example of Usability Testing at Google

Participate in Google User Experience Research Studies

Related search:

Improve this answer
8
  • 3
    There are some who say that the process used by Google isn't design so much as selection... Like with their search site, they design a couple of alternatives first, then put it in front of their audience and track which "works" better. In other words: selection.
    – Marjan Venema
    Commented Oct 27, 2010 at 6:15
  • 1
    @Marjan Venema: What is design then? The user always makes choices by selection, and Google is creating all the choices for selection, they're not magically appearing. Selection based design is not the same as organic evolution, where the only designer is the system itself. I'm open to understanding you're point, just having a hard time understanding.
    – blunders
    Commented Oct 27, 2010 at 11:44
  • Design is the process of coming up with a solution to a problem, balancing the, often conflicting, forces acting upon the object under design. The process can come up with alternate designs, each the result of giving more or less precedence to one or more of the forces. You may find this interesting: 37signals.com/svn/posts/… The q&a session video on the page addresses design v selection, it discusses a/b testing and the 42 shades of blue "design" by Google.
    – Marjan Venema
    Commented Oct 27, 2010 at 18:04
  • Btw, I am not saying Google doesn't do design. All I am saying is that the testing and studies Google does (as mentioned in the answer) are more about selection of the solutions they have already designed...
    – Marjan Venema
    Commented Oct 27, 2010 at 18:07
  • @Marjan Venema: Would you say that all user testing is a science, or that there is an art to it?
    – blunders
    Commented Oct 27, 2010 at 18:55
2

The reason that "According to Spool, in the last 30 years, there has not been one website or other digital innovation that can point back to usability testing as the defining factor for its success." is that usability isn't that important to commercial success.

Usually the product with better marketing wins, not the product that is more usable or technically superior, shocking, isn't it?

People are not motivated by usability - when was the last time you've heard someone say "The new accounting system is so hard to use that I prefer to not get paid"?

There are two major points here:

  1. People will overcome usability issues to accomplish their goals (otherwise nobody would drive, just try to think about usability testing a manual transition car with people who don't know how to drive - you'll have close to a 100% failure rate).

  2. People buy things because they help accomplish goals, make a good impression on other people (usability of woman clothing anyone?), stroke your ego (status symbols) help elevate fear (alarm systems, insurance, etc.), because everyone else has one or because they look so unbelievably cool in demos you just can't help yourself (Apple products).

Usability of a product can make a significant impact on the user's quality of life but has only negligible affect of buying decisions.

Improve this answer
3
  • +1 I dream of the day I can actually buy good-looking AND usable kitchen hardware... (right, right, some exist today but only specific things like a particular brand of single-serve coffee machines - not generic integrated stoves, ovens, microwaves, dish washers and so on, because no-one buying them seems to care (until they try to use them, get upset and then feel dumb because they think this is the way it's suppose to be). My microwave has a grid of identical buttons with poor icons that need to be pressed in a certain order before the knob is even allowed to enter a time...
    – Oskar Duveborn
    Commented Oct 30, 2010 at 13:46
  • @Oskar Duveborn - I understand you, just this morning I was thinking to myself I should have written here about my baking oven - it usually takes me 2-3 attempts to set the timer on the thing, but for baking it's the best oven I have ever owned and I would buy it again without thinking twice (because good bread beats usability, obviously).
    – Nir
    Commented Oct 31, 2010 at 12:04
  • True, if it does what one need and does it good then some minor quirks in usability is acceptable until beaten. At least until you get to professional usage levels I'd say. if you had to use it continuously over each day perhaps it would become too much of a nuisance? (or you'd become an expert on the timer sequencing) ^^ I had an important example to make but someone distracted me so I forgot it :9
    – Oskar Duveborn
    Commented Nov 1, 2010 at 8:42
2

I hate usability testing, HOWEVER, I do know of a product that answers your question.

Quicken came out first in 1984. (History) They did usability studies (called "follow me home") and discovered that people were using Quicken for their businesses. So Intuit developed QuickBooks in 1998 which specifically addressed that market segment. QuickBooks would never have been implemented without the Quicken usability studies.

I still hate doing them.

Improve this answer
1
  • 1
    Additionally, they tracked the amount of time it took a user to write a check from first opening the package. This was an example of a usability success metric, and arguably helped Quicken beat out its many early competitors.
    – Jonathan
    Commented Jul 7, 2011 at 0:49
1

The purpose of usability testing is not to create a great user experience, but to remove the interface from the user experience.

When the wife of a business friend - who works close to that field - said many years ago "the best UI is no UI at all" I did a double take. At that time, I, too, thought the ideal UI would wow and bedazzle users.

But it resonated with me: "No UI" meas the user doesn't notice an interface. He doesn't push X to warp Y, he just warps Y.

UI testing would usually optimize towards reducing mistakes, repetitions, and having the right option just under your mouse. A great user interface encourages exploration, and creates pleasure using it. Many simple flash games are a great example of that pleasure.

The author of the Spool slides seem to have worked with quite some cargo cult UI designers - people that followed the rules, but don't understand why they are there. You can butcher every job with that. Unfortunately, the slides don't give any proof of his statement that "user centric design never works".

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.