Your solution is not bad at all and you incorporated Law of Proximity (1) pretty well.
My only suggestion would be to to combine this with Law of Unified Connectedness (2)
You've mentioned that that:
Within this main group, a maximum of 100 sub groups can be active
- Grouping categories with outer border.
- Addition of number of nested items per group.
- Linking categories to parent category by adding border.
- Moving dropdown arrows closer to the titles. It should be easier on the eye for end user.(3)
(1) What Is the Law of Proximity?
The law of proximity describes how the human eye perceives connections between visual elements. Elements that are close to each other are perceived to be related when compared with elements that are separate from each other.
The law of proximity allows us to use whitespace, for example, to build perceived relationships between different elements.
(2) What Is the Law of Unified Connectedness?
The law of unified connectedness states that elements that are connected to each other using colors, lines, frames, or other shapes are perceived as a single unit when compared with other elements that are not linked in the same manner.
This grouping effect works even when it contradicts other Gestalt principles, such as proximity and similarity. “How can it contradict laws?” you might ask. Well, remember that we’re dealing with the unique combination of the human eye and brain. Without getting into the subject of magic, illusions or tricks (because we don’t want to get off topic), we can see that the human tendency to link or group elements, or focus on like items in a sea of dissimilar objects, is a powerful trait.
Let’s try a quick experiment. If you have another blank page, please take it and try this little sketch. It will take a matter of seconds.
Draw six, rough circles, like the six dots on a pair of dice. Now, draw a line from the top-left corner dot to its comrade dots at its right and below it. For the bottom-right corner dot, connect it to the dot above it and also the bottom left dot. Look away for a moment; then, check out your sketch. You now have two groups of three, linked dots.
Implementing this connectedness in your designs is easy; there are many ways to indicate grouping within a design. Some common examples include:
- Connecting related links or buttons by adding them to the same drop-down menu.
- Using the same bullet shapes, colors, or numbering system (such as Roman numerals, Arabic numbers, etc.) on list items to group them with each other.
- Displaying functions of a similar nature, such as login, sign up, and forgotten password, so that they are related, inside a frame or colored rectangle.
(3) Those are all assumptions and would need to be validated and tested with the end users