A lot of hard evidence, literature, and references can be found by starting from this article on searchtools.com. Some excerpts follow. The gist of it (according to my understanding/opinion) is this: a user interface should enable the user to do exactly what they want to do in the most efficient way possible. If the user is going to want to search for something, give them a search tool. That being said, I agree with you 100% that a search is not something you can simply tack on to an existing UI. A lot of thought should go into how it is designed, with the ultimate goal of helping the user find what they are looking for. Google's PageRank is an excellent example. Users don't care about how many keywords some amateur web designer can stuff into a page, so traditional search methods are mostly useless. By tailoring the search algorithm to give higher rankings to popular and well-referenced sites, Google comes very close to matching exactly what the user wants.
Most of the references below are pro-search, but I think the following cartoon really gets to the heart of the matter. A search feature by itself is not enough.
It is important to note that foraging for information does not equate to aimless "surfing." Foraging refers to the variety of strategies seekers exhibit in their quest for information and how humans adapt to their environments on a situational basis. Consequently, in an information-rich world the real design challenges are not only how to facilitate finding and collecting information, but how to optimize the seeker’s time.
According to Chi's research, people almost always start out with a search engine, then engage in what he calls "hub-and-spoke" surfing: They begin at the center, and they follow a trail based on its information scent.
One of the surprising patterns is that much of the searching shows clear evidence of 'jump search' - using the search box as a convenient shortcut to known content; for example, about 10% of searches are looking for a course by its code number.
Users now have precise expectations for the behavior of search. Designs that invoke this mental model but work differently are confusing.
Search is such a prominent part of the Web user experience that users have developed a firm mental model for how it's supposed to work. Users expect search to have three components:
- A box where they can type words
- A button labeled "search" that they click to run the search
- A list of top results that's linear, prioritized, and appears on a new page -- the search engine results page (SERP)
In user testing, people tell us that they want search on websites and intranets to work like X, where X is their favorite major search engine. Luckily, all three of the major engines (Google, Yahoo, and MSN) work the same: exactly as stated in the list above.