Calum, I see two challenges with nested modals:
- Users may lose track of where they are, and
- Backtracking through multiple modals is tedious.
I've seen quite a few designs that go two, maybe three, modal dialogs deep, and they worked reasonably well, because those modals were highly focused.
For example, here's a screenshot of macOS's Siri preferences panel:
Clicking Siri Suggestions & Privacy… opens this modal:
Clicking About Siri & Privacy… inside the modal opens another modal that appears above the previous one:
So, to return to the actual preferences panel, you'll have to dismiss two modals.
Compare that tothe DNS configuration in an older version of MS Windows:
All of the five windows you see in the screenshot are modal dialogs. In this case, users will likely have no clue where they are in the chain of dialogs, and getting back now requires manually dismissing all five dialog boxes.
Nested modals are never ideal, but in some cases they may be difficult to avoid. If you do need them, make sure that users understand what path they follow, and also take into account how often they need to follow that path: if the associated task is performed often, nested modals might be forbiddingly tedious; if it's a rare task, users might not even consciously notice that they are navigating nested modals.
All that said, have you tested your design? If not, how about creating a prototype — a simple paper prototype might already do the trick — and doing some simple "hallway testing" in your company to get some baseline feedback?
Good luck! :)