As far as I know mixing solid and hollow icons is a bad practice. Why Apple does it then in iTunes desktop software? Are there any iconography principles that allow mixing of these two icon styles? On the screenshot below icons such as "Music", "Books", "90's Music" have solid fill, others are line type icons.
In general, I think that from an aesthetics standpoint it's reasonable to combine them. What Apple does in the list is a question of readability. Note how the "Books" and "Audiobooks" share the same shape, but are filled/outlined. They are next to each other, and using a filled icon for Audiobooks with a play symbol would be more consistent, but it would be less readable.
The Nielsen Norman Group research on Icon Usability suggests four criteria to evaluate icon usability:
- Findability: Can people find the icon on the page?
- Recognition: Do people understand what the icon represents?
- Information scent: Can users correctly guess what will happen once they interact with the icon?
- Attractiveness: Is the icon aesthetically pleasing?
Considering that list, Apple's list focuses on findability and recognition, rather than focusing on Attractiveness and consistency and thereby ignoring the ever so important first two aspects of that list.
In my personal practice, I quite often use hollowed out icons for non-imperative/important calls to action, e.g. buttons which are not filled in, but have a border, while I keep filled in icons for larger, more important calls to action which have better visibility in terms of color, weight and presence.