I've tried a variety of techniques (and I'm excited to hear about new tools & methods from others), but our current go-to approach is to first run a quick audit using Website Auditor (yes, the website is kind of scuzzy-looking, but the tool is good). This app crawls your site structure and delivers a nested hierarchy view, including titles, URLs, meta tags, redirects, broken links, etc. It can also do more advanced things like HTML code errors and Page Rank, but we don't use those very frequently. Currently, we're using this as our reference data, and then hand translating it into a visual sitemap in OmniGraffle, but I'm sure you could develop a more automated process.
To make sure we don't miss anything, we also go through the files themselves via our content management system -- but then, I'm an in-house person, so I likely have more access to such things than an agency would.
Alternatively, I've used Site Sucker for Mac (which is free) to download a local copy of an entire site, and used the log file as my content inventory. However, it's not as useful because it doesn't generate a nested view nor any of the other niceties previously mentioned for Website Auditor, so it's definitely a "poor man's tool".