Of the options you give, I’d say your best bet is to pursue multiple top-level primary windows. I’m not aware of users being confused by an app being spread over multiple primary windows. MS Outlook seems to do okay with this approach where individual letters and appointments appear in a separate primary window from the “home” window. You can still have menu items that affect all windows (e.g., application exit, and a Window menu, which may include Iconify All), so you aren’t losing capability. MDI’s windows-in-a-window complexity has several serious usability disadvantages. Maybe it’s time to ditch it.
Instead of relying on a container window to comprise your primary windows, you can apply some graphic design to visually tie the windows together (e.g., give the window borders, menu bar and toolbars a non-theme non-standard background color). However, it may be sufficient to have a common icon and program name in the title bar. Also, there’s something to be said for the users not seeing all the windows as a single app, but instead just seeing “documents” on their screen regardless of their respective apps. This helps users to flow freely from window to window (mentally and spatially) as the task requires, including among windows of different apps.
In your case, one problem with multiple top-levels may be that it’s easy to lose the smaller windows in the clutter of the desktop. Also, if users typically have a lot of such windows open at a time (five or more), then the task bar can get awfully crowded. You may want to cut down on the number of small primary windows:
Combine small windows together. If several of your small windows are related to each other, you could put them in a single window. This could either be a large window with divisions between the content. That won’t get lost and the user has immediate access to all the content. Or it could be a small tabbed window so it doesn’t get in the way.
Use modeless dialog boxes. If there is a hierarchical relation among your windows, you can turn your small windows into modeless dialog boxes or utility windows. Dialog boxes and utility windows always float above their parent primary windows, so they are less likely to get lost. They lack an icon on the task bar, so it doesn’t get too cluttered.
Combine small windows with a large window as panes. Another approach for hierarchically related windows is to join small child windows to a large parent window. For example, a main content window can include adjacent panes for Layers and Properties of content in the window. Alternate views of the same content (e.g., a small “overview” map for the main content) can also be in the same window divided into panes. This is often better than separate small windows (including dialog boxes) because panes won’t occlude the main content like small windows can. The panes should be closeable and resizeable so users can adjust the space dedicated to the main content.
Docking panes/secondary windows. You can have both of the above options at once if you allow the users to dock/undock the window/panes to suit the current needs. This is common for palettes. You only have to decide whether the default should be a pane or dialog.
In other words, the real solution may be a combination of the options you list.