We are setting out to design a system capable of managing a system that is comprised of 100,000 individually addressable resources (electro-mechanical controllers).
One very important visualization we need to create will be geographical. Operators need to be able to navigate around the resources with some spatial awareness.
The old system we built only needed to support 1,000 resource so it was built using a single map screen where everything could be seen at once - kinda brute force. This pretty much worked, but we do not think its going to scale to 100,000.
Not only that but 100,000 poses lots of other "bandwidth" problems, not just rendering. We need to consider communication bandwidth and cognitive bandwidth.
Not to mention safety is important in this system so errors and warnings need to be brought to operators attention in some way that they are not overloaded.
So far we think we need a tree structure, where data is aggregated at each level and operators can drill down. This is great in terms of "bandwidth" but the problem with a tree is it takes a while to get down to low level and also resources that straddle branches cannot be visualized at once.
So we also think there needs to be some "search query" type UI for quickly finding certain resources. OK great, but the search could end up matching 100,000 resources so we are back with the same problem. (We need to show the search results in a map too).
Even though (obviously?) this is not a web browser applicaiton we are using the web paradigms as much as we can (tabbed browsing, back/forward nav, text URIs etc).
Do you know of any case studies of systems of this scale?
Do you have any thoughts about how you would go about designing a UI for managing such a large set of data?