These are very popular quotes from Steve Jobs and are frequently pointed to as reasons why we shouldn't test. If Steve did it, so can we!
The trouble is that most people miss several key points. The first is that they are not Apple:
A Forbes article, Five Dangerous Lessons to Learn from Steve Jobs, lists these quotes as #1. Chunka Mui, the author of the article, calls it out:
I actually think Jobs was right but only in the very narrow category to which he aspired: where his products, such as the Mac, iPod, iPhone and iPad, either redefined or created product categories. That’s not the domain in which most businesses play. Remember also that Jobs backed up his unique insights with an enormously expensive creative process populated by world-class designers. Without Jobs’ talents and the unparalleled creative team and processes that he built around himself, you won’t get away with doing no market research and not listening to your customers.
Another is to confuse statements about market research with user testing. Apple does do user testing!
In 2010, an iPhone 4 was found in a bar. Thing was, the iPhone 4 didn't exist yet - as far as the general populace was aware! It certainly wasn't left there on purpose, but instead likely by an employee who was testing it.
Another key point is with wording: want vs. need
People/users know what they "want". They "want" a lot stuff. They often times don't know what they "need"!
Steve Jobs understood this. In another article, The Steve Jobs Market Research Quote Should Rest in Peace, two different types of market research are discussed. It is clear that Apple does conduct market research, but they ask the right questions -- they look at what people need out of a product, not what they want out of it.
(Note: I'm not implying people "need" an iPhone, but what they "need" out of a product is very different then what they "want" out of it)
Broadly speaking, there are two types of research that work--and they are completely consistent with Jobs' words and success. Market Research that Jobs Agreed With First, you must spend time understanding and gaining insights into consumers' existing habits, beliefs, routines and unmet needs. We would never ask people, "What should we develop to make your life easier?" Instead, we spent time with people in their homes and watched them at the store, then we dug through data about what they are buying and using. Your job at the early stage of innovation is to get into the shoes of your customer, understand her life, and look for insights that give you ideas on products that you could create that would surprise and delight her. And this is exactly what Jobs was so good at. He saw early on that people were interested in using computers, but were frustrated at their complexity. He saw that people loved music, but couldn't stand loading a few dozen songs at a time onto the early MP3 players. Jobs deeply understood how people engaged with technology.
The site UXMyths hits on this as well: Myth #21: People can tell you what they want, which has a quote from Jakob Nielsen (among many other excellent example):
Jakob Nielsen says that the “critical failing of user interviews is that you’re asking people to either remember past use or speculate on future use of a system.”
To some extent, Myth #29: People are rational also hits on this point.
A last point, people always point to the iPod, the iPhone and the iMac as examples of why these quotes from Steve Jobs show "user testing doesn't matter." So, let's look at all the other awesomely successful products Apple has produced:
For some reason, people forget how often Apple (and Steve Jobs) has failed; and there are a lot more examples then just those above.
Question is do we need to test, or don’t we?
Yes. Yes we do... and we can use Apple as an example of why we should! For two main reasons:
- Because Apple does test their products
- Because sometimes when you introduce something so new it flies, but more often it fails.
To extend on the "failure" concept, per a few comments below. The Newton was created sans-Steve, as was the Pippin. So, here are a few more products under Steve's watch to fill the gap of those two:
My point here is that Steve/Apple have failed plenty of times, some past and some more current-ish (G4 Cube). So to put any quotes by Steve as evidence to, or not to, do something because of the recent iPod/iTunes/iPhone success is ill-advised. They absolutely got it right, this time.
I point above that Steve/Apple does do testing, they do perform market research, and Steve also built a very talented creative team in Apple's later year that didn't exist in the beginning. But the "magic" doesn't always happen.