Take the 2-minute tour ×
User Experience Stack Exchange is a question and answer site for user experience researchers and experts. It's 100% free, no registration required.

This is probably wishful thinking but here goes...

Now obviously I don't expect some magical app to crawl a website, summarise each page and detail all the assets within each page (if only!) however is there a crawler of some sort that will trawl a given website and produce a full set of links and respective hierarchy would be invaluable. That would ensure that no pages get missed out as they invariably would through a purely manual inventory.

From there I would be able to see the existing site hierarchies and taxonomy decisions that have been made when building the site, and could use this as the basis for creating a detailed website content inventory.

Whether this be an online application, script or even a useful manual technique other than just manually sitting at the site and following links would be great.

share|improve this question
add comment

closed as not constructive by JonW Jun 26 '12 at 13:07

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

1 Answer

up vote 2 down vote accepted

Website Auditor is a tool that crawls your site and delivers a nested hierarchy, complete with meta info, broken link information, error codes, etc. We've found it VERY useful for the exact purposes that you list.

Alternatively, I've used Site Sucker for Mac (which is free) to download a local copy of an entire site, and used the log file as my content inventory. However, it's not as useful because it doesn't generate a nested view nor any of the niceties previously mentioned for Website Auditor, so it's definitely a "poor man's tool".

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.