There is a study that users do not go past top 3-5 pages in a Google search, on which predominantly everybody would agree. If you would get your best results - it means to be within the top 5 page at least. Merely users abandon to search or change the query if the results do not get laid in the first few pages. In that case, why do we have MORE search results, pages 10, 11, 12, ... and beyond? What meaningful impact does it serve when users rarely do get to that page? Is there any useful content lying far deep?
You have to realize that Google's or Bing's job as search engine is not worry about the behavior of their users with regards to how deep they are willing to dig to get results.
Their role is provide relevant search results based upon the keywords entered and the matching SEO of the site.Though Google (and Bing) use algorithms to determine the optimal search results based upon the keywords entered, the success of the search results depend on two factors
The keywords entered : Though Google does try to help out by features like Google instant,spell correct and autocomplete,it cant account for all unique cases for which the user is searching for and if the users keywords are not specific enough, he might not get the search results he wants
The SEO of the site : If the site has relevant content but doesnt have a decent SEO you might not find it at the top of the pile (within 10 pages) especially if its about a common topic
You also need to account for the fact that the popularity of Google and Bing depend upon the number of pages they index and hence they need to show as many search results as possible.
Another reason Google provides the full number of search results, sometimes in the millions, to provide the user with an idea of how much more the search query would need to be refined. This encourages the user to make better use of the engine by making them do smarter searches and they account for people using incorrect keywords or having very specific or ambiguous requirements. However they do restrict the number to 1000 results as going beyond a thousand would eat up their resources because they have to rank each hit, and crawling the Net every day, several times a day, makes that unrealistic and unnecessary.
Bottom line: Its not really about UX,but its about showing people that there is a ton of information out there about any subject they choose to search for and Google/Bing can help them find it for them and also provide incentive's to users to perform smarter search results after a while
That said here is a nice article on how to build an effective search results page
Why not show them? The data is there, and on the first dozens of pages it's still pretty relevant. In terms of UI it doesn't cost you anything because you're using the same paging control that lets the users navigate through the first pages. If you tell your users that you've got millions of results but you only let them view a hundred, it makes them wonder why. For an "arbitrary" cut-off, 100 pages is very reasonable, while 10 or 20 is a bit strange.
Search engines such as Google or Bing devote their resources to having high-precision queries for common, precision-oriented information needs. Think of them as recommender systems for popular documents. This works great when you're interested (as many people seem to be) in what Britney Spears is up to, for example.
If, however, a user's information need is recall-oriented, then settling for one of the top few results may not be appropriate. Think about searching for symptoms related to a disease, doing research, or trying to make a new connection between seemingly unrelated concepts. These kinds of searches may require people to dig into the results more deeply.
Users often do not find a good result in the first page - this depends on how specific of a result they are looking for and how many false matches fit their query.
I personally often go through a few pages of search results before refining the search query to try and improve the quality of the results.