I know this question is extremely broad but I would like to know the steps a UX'er would take in order to understand what content there is on a site and then how to organize this content in terms of navigation. How do you know which content should be grouped with what? How do you know which items should be in the main navigation?
I think the question you should be asking is what should be my content strategy and how should I define that to drive my site navigation..
That said, there are multiple steps in defining your content strategy :
Plan your content:
This is the initial phase of any site and should involve the questions such as the objective of the site, the user group which you are targeting, the primary audience,the secondary audience and so on. This is the most important part of your content strategy since it defines your entire site flow and also your navigation. Content planning has the following steps :
From the previous stage you would have got a good idea of the content you have with potential user flow. Use this information to map the content by creating wireframes or content maps which will help define the interactions between content flows and also help define the user navigation such as: User lands on home page --> clicks on banner --> clicks on product --> adds product to shopping cart --> Goes to the cart page and checkouts --> makes a payment --> is presented with a confirmation message
I recommend looking at this article on additional inputs on the benefits on creating content maps and using them to define your navigation flow. To quote the article
Present your findings
In this stage present your preliminary findings and wireframes and content maps to your design team to highlight the navigation flow and what influenced your design
Here are some additional articles for you to read
I've solved this task using card sorting in a low tech, hands on session with my client and/or users.
Write down all menu items onto index cards and let the participants sort the cards in a way they think it is correct.
You will find a lot more information on the internet: http://en.wikipedia.org/wiki/Card_sorting
Understanding the content
To be able to understand the content currently available on the site you can't only rely on the navigation menu, tagging systems or search which is where you start. Get your view of where content lives before you ask the users and editors of content today. You need to interview and observe content editors and content consumers since users say one thing and do another. Questions to ask is how they find what they need (navigation, tagging, search) and if they archive content somewhere else after a period of time, after project closure or some other time or task interval. Don't forget to ask how they organize content today when they creates and edits content. It will give you an idea of how they work today.
When you have a clear picture of where content lives, how it is structured and classified, and for how long it's time to document and schedule a workshop. Invite users from different departments and different roles and have a workshop on content management, and see if what you discovered (in your documented report) is the same as the workshop tells you. If it is - you can move ahead.
How to organize this content in terms of navigation
As many have told you already, the best way to organize content is through Card Sorting. You can use open card sorting or closed card sorting. I would suggest you start with an open card sorting where users are asked to organize content together and label it. That way you get the correct terms the company culture are using. It saves you a lot of work as you move along if you use the same terms your users are used to.
You would probably need a couple of iterations of card sorting sessions where you can end with a closed card sort. A closed card sort have predetermined labels which users are supposed to organize given content into.
Which content should be grouped with what
From the card sorting sessions you have bot the labels and content that goes within the label in a nice hierachical structure. But don't make the mistake of having to strict hierarchy since users need to find content related to each other which hierarchical doesn't fit. Those graph edges are your relational/supplimental navigation which usually is a part of the hierachical navigation structure but are not bound to it. A level 4 element could very well link to a level 2 element as a relational link.
Which items should be in the main navigation
The elements following the root node of your hierachy should be in the global navigation. And if you have done your content analysis and card sorting correct - the global navigation should be very obvious and easy for your users to understand.
Usually there are client objectives like, "we need to show the following features first", provided in the brief or discussed in workshops.
Then there are user objectives, "I want to find out where to buy this", derived from personas based on user research.
A balance of these two factors should influence your first attempt at creating a nav. The best way to get agreement is to get data from both clients and potential end users is in a card sorting exercise.
But a physical stack of cards and 5 mins of time works just as well.
Specifically addressing the need to understand the content, there are two steps you probably need to go through if you haven't already.
The first is to do a thorough content audit, so that you're aware of the breadth and depth of the content you're going to need to organise. Navigation solutions are likely to vary dependent on the volume of content you have to sift through, and in addition performing an audit may allow you to pattern-spot, suggesting some natural categories or hierarchies you can work with later.
The content audit should not only include a full listing of the content available, but should probably also pull in whatever usage data is available (server logs, analytics) so you can get a good idea of which content is currently being used - which might suggest some measure of importance you can use later when determining priority.
The second is to work with a subject matter expert. The major advantage of this approach is that you can get an understanding of the content from a holistic perspective - which pieces of content are important, as well as which pieces fit into typical tasks that a user might be expected to accomplish. A SME can also highlight various different ways that the pieces of content might be related, though you need to take care not to adopt their mental model entirely (which may be subject to bias due to their familiarity). SMEs also have the benefit of their domain knowledge, which may be useful for actually understanding the content itself (potentially a roadblock in a technical field).
Becoming familiar with the content in this way will hopefully allow you to make better decisions when you come to building your navigation.
Have the goal in mind of being able to get to valuable information with as few taps/clicks as possible.
Optimise for those who enter through the inner page and start to explore your website from there...
Learn how the end users would sort and group categories/content as someone suggested above.
In this area Lou's book is most valuable http://shop.oreilly.com/product/9780596000356.do
I have organised many different sites and have learnt a few things from that process, some of which are counter to what is being said by others.
To organise a site you must first know all the content you have, a full content audit and 'as is' site map generated by hand is vital. You can automate this process to generate the end result but this skips the vital step of understanding the existing content.
Secondly you need to understand what tasks the users are doing and how it relates to what they do in real life. Talk to real users and draw up user scenarios that explain what they do. Use site analytics to see what people are currently doing (if there is a current site). The site should be organised according to task EVEN if it is a content based site.
Internal workshops are useful to understand what the business want - it's vital to get a clear set of site objectives. Prioritisation of functionality can be done using post it notes and people sketching up 'home pages' and telling stories.
I don't use card sorting and have found its output to test badly when a site has to be used. Often it creates sites that a clumped around similar sounding things, but not real usage patterns. For that and other reasons I would not recommend using it as a primary way to uncover the Information Architecture for a site.
I would recommend using tools like TreeJacker to test pure navigation structures with end users.
Because you're redesigning a site and not putting together something from scratch, I would not merely rely on your own experience/impression of the site. I think there are a few key tools everyone should use when redesigning...
1) What links should my navigation consist of?
If your site is content-driven and serves advertisements as its source of revenue, then it's all about content. There's not likely to be a target page you want the user to end up on, like you would on an ecommerce site. With this in mind, for a content-based site I would focus on what the Top Content analytics data says and order my content by popularity. For an ecommerce site I would use Google Analytic's goals to measure the effectiveness of different routes customers take through my site which leads them to purchases.
2) Where is a good place for my navigation?
Google also offers In-Page Analytics which can help you determine the most effective UI elements within your web site design. Changing the color of a navigation item or link can make a big difference. You can measure that kind of thing using this tool.
Most of the answers have covered the strategy for designing the information architecture with respect to how a user might navigate through the information. I would just like to add a point here that relates to adaptive content design (Karen McGrane talks a lot about this). Because there is also the issue of making the content flexible and adaptable enough that you don't have to tear up your navigation everything the content of the website changes, and this should be a consideration in the redesign of the navigation as well.
I like to stick with data first, and augment with human behavior through heuristic evaluation later.
I do this because when dealing with a team - and supervisors that have a stake in the project - how information is organized is often handled emotionally. Certain VPs and SVPs like to see certain things at the top because it "looks good". So using data means that the emotional part can be taken out of the equation - sort of. Bear with me.
Data first. Analytics will tell you what people click on the most, and will help prioritize the main items. The downside of this is that people might click on those things the most right now because of poor, flawed or incomplete navigation.
Ref: O'Reilly Information Architecture Book (Polar Bear book)
So next we find out what people are SEARCHING for on the site/app. This can help to determine what people couldn't find and therefore needed to actually SEARCH for.
Both of these methods still have some weaknesses. Many studies seem to show that there are commonly two types of people - those who SEARCH and those who CLICK.
After these things (though some might say BEFORE these things) your team/company needs to determine what goals/actions you want people to do when they visit your site/app.
So now you have data on what people are and are not doing on the site. You have your orders from the boss on what people need to do on the site.
At this point you can introduce the card sorting techniques with the team. But now, armed with your research, you can help your team (and higher ups) see what users are looking for and what your company needs them to do.
Ref: [card sorting links above]
Now that you have navigation in place, it is time for some heuristic evaluation. This will iron out the details.
Now you tweak and go live. (In a perfect world! Yeah! Good job!)
I would then suggest going the extra mile and setting up funnels in Google Analytics (or the tracking mechanism of your choice) and make sure people are reaching their goals.
And finally, I feel this is analysis that never ends. This might be me personally but I believe this is something that deserves a small amount of attention on a regular basis to ensure your navigation success.
I could give you a simple rule: try to make the navigation hierarchy system\links such that every part of the website could be accessible with at the max 3 links\clicks.