In a broad sense, I can safely say this is certainly not the case. You're on to a little something marketing-wise, but first I want to make it clear there's plenty of great visual design that lots of everyday people are hardly "turned off" by.
Look at the major operating systems; OSX, Windows 7 and iOS are extremely visually polished. iOS and it's fancy design has been advertised from the start as something easy to use (a reputation it largely lives up to) despite looking all neat and pretty.
There's even a great amount of precedence that shows polished design is considered more usable. Research has backed this up as Jimmy Breck McKye points out. As some examples, look at iOS; from the start it's been much more visually polished than Blackberry and Windows Mobile, but it was immediately received better by non-technical audiences compared to older smartphones. Ubuntu is one of the most polished Linux distros out there and widely considered one of the most usable. Older versions of Android were quite bare-bones visually and had significant usability issues; more recent versions are clearly both more usable and more visually polished.
It's quite clear that usability can be improved alongside visual polish, and it's even been shown in research that a similarly functioning device will be considered more usable than the same device just "uglier" due to the Aesthetic Usability Effect.
What I think you're really hitting on here is an issue of branding.
Let's take a convient if fictitious example:
Guns are from Borderlands 2 by Gearbox Software
Which guns do you think cost more money? The Maliwan guns are clearly fancy, high-tech, elegant, and probably expensive as heck. The Tediore guns look like they may as well have a Walmart logo on them; cheap, disposable, ubiquitous.
This is common enough in all forms of design; compare a Honda Civic of any given year to the matching Mercedes SLS AMG. I don't think many would actually look at the two cars, both at the same price and say "I'd prefer the Civic"; the turn off from the Mercedes is that it costs ten times as much. Through experience we instinctively now know that those fancy things probably cost a lot.
This psychological effect has roots in the physical world; it actually does cost a heck of a lot more money to make a Mercedes SLS than it does to make a Honda Civic. The interesting thing with digital design is that while it may cost more money to design nice looking software, it doesn't necessarily cost tons of money to the consumer. Beautiful websites are delivered to hundreds of millions of people for free every day.
That brings me back to Windows 7. Windows 7 does cost a fair bit of money, but it's hardly a luxury item. The look and feel does turn some people off (there is no such thing as a universally appealing design, period) but in the digital world, good looks are not reserved for extremely expensive things, though a slick design still fits a brand image just like a cheap-o simple design fits a cheap-o simple brand image.
What's really off-putting is the appearance of exclusivity. Windows 7 has never been branded as something "exclusive". Mercedes is, Maliwan is. There's a lot more to making something feel exclusive than a good visual design; high prices, members only, devotion to loyal customers over welcoming new ones. These are things deeper than the visual polish, and they're not necessarily "bad" either; Mercedes exclusive image is fine because most of the people they turn off wouldn't actually purchase the vehicle, and at the same time the exclusivity is fun for those who do.