Progressive disclosure is what you want
By knowing which controls are often used and which are only rarely or almost never used, you are already done with the important part.
Removing screen elements that are infrequently used reduces clutter and information complexity and makes the remaining content count more.
The "Show more" button is a common control to deal with this. I don't even have to look far, UX Stack Exchange has one right in the comments section:
YouTube has multiple of those to keep the UI clean and less complex:
You will need to find a sweet spot for what information to put inside the invibile space and what to put outside of it, being always visible. People who do not use the additional controls will never see them. Those who will use them eventually will go through a short learning phase, discovering the "show more" button, clicking it and then finding the control they need.
You are supporting discoverability by putting the button right where users would look for the "missing" information, so next to or under the available information should be ideal. Using a hyperlink or accordeon style helps afford clicking.