Overview
Looking at the innovations made by Apple and HTC as Apple moved from iPod to iPhone and HTC from personal organizers to mobile phones gives some insight into the design process. It's necessarily speculative, until someone from the inside gives a definitive answer.
- Test bias can explain a lot. If you don't test the pinch-to-zoom gesture, dedicated buttons for zoom will look better than if you do. You won't test that pinch-to-zoom gesture, if you're still thinking 'resistive screens'. Resistive screens are mono-touch.
- Test bias can explain a lot. If you never test apps together, only in isolation, you have a problem a bit like the problem with the sip test described by Malcolm Gladwell. A test that looks at UX one app at a time is more likely to determine that each app needs more buttons.
Detail
But why are HTC and other phone
manufacturers deviating from that
first design?
It's NOT a first design.
Using 'too many buttons' is mostly down to history in the two companies and the way designs evolve. Apple moved from a low cost non-touch-screen MP3 player to phone. HTC moved from a premium non-touch-screen personal organizer to phone.
Why did Apple shed buttons? Apple pay attention to cost reduction early in the design. It was particularly important for iPod. They got in to these capacitative touch technology/chipsets via iPod 'wheel' which was largely motivated by reduced manufacturing costs. A capacitative wheel gets more input flexibility for a much lower price than lots of quality buttons do. Unlike a resistive screen capacitative touch is multi-touch. It's a good tradeoff, using smart electronics to replace mechanics. As well as reducing cost it's also good for reducing size and weight. From their experience with click-wheel Apple knew that touch screen could make front buttons redundant. The one front facing button is just the last remnant of the click-wheel. Their click-wheel experience, and cost reduction, guided them on their path to minimalism.
Why didn't Microsoft/HTC shed buttons? They were building on their own history and experience. The idea of having an abundance of buttons is the norm in for example HTC's iPAQ (five front buttons). Their existing software relied on it. Is it so surprising that their usability studies didn't pick up the opportunities for fewer buttons? Users can only rate options that are put before them. They used versions of their existing software. Microsoft/HTC engineers were thinking in terms of clicking not sliding - whilst Apple engineers had already gone beyond the sliding in click-wheel to multi-touch. If you've got multi-touch you don't need a zoom button. This is the crucial point. The low button count options Microsoft would have presented to user testing would not have been as good as Apple's, and in particular would have been less good at using sliding. The answers they would have got from user testing would have told them clicking was important, and more buttons were better.
The history argument also answers why Apple put that button on the front at all. It isn't needed there! With the right software the only buttons that are needed still are one's that are used when the screen is dark, and they can and should be side buttons. Tablets are going in that direction.
Edit
A related test bias issue: User testing one application at a time will introduce a bias in favor of dedicated buttons. It's only when you're using your calendar that a dedicated 'record' and dedicated 'send' button that aren't being used starts to niggle. This is where vision helps. Having an overall vision for the product you are more likely to test and refine the vision as an integrated whole, rather than blindly build a solution by combining items tested only in isolation. The vision thing is a cultural difference between Apple and Microsoft.
I've said that Apple offered better options for touch for users to try, and hence saw better user satisfaction in testing. Apple's killer touch feature was of course zooming using multi-touch pinch gesture. If, as Microsoft did, you have mono touch you can't do that, so on mono-touch their user tests would rate a button to zoom-in or zoom-out more strongly than in Apple's tests. Multi-touch didn't come to Windows mobile (HTC) until early 2010. The pinch gesture became an Apple patent in 2009.
Cost Reduction
Keeping manufacturing costs down is only one of several competing driving forces in design. Sometimes the trade-offs swing in favor of the cheap option - a built-in battery is cheaper smaller and lighter than a replaceable one - fewer buttons are cheaper - black is cheaper - because a black background extends battery life. It's not minimalism or aesthetics that caused the switch from white to black. It's not minimalism that advertised the built-in-obsolescence of yesterday's iPhone by giving it a built-in battery.
Sometimes the trade-offs swing the other way. Capacitative screens are more expensive to make than resistive. Capacitative is more responsive but less accurate than resistive. They're worth it because they enable multi-touch, a better UX and fewer buttons.