This depends on the amount and type of content you have.
If you are expecting a lot of items (i.e. reservations) in your list, you should show the empty spaces. If you have few items, you should only show the filled spaces.
The more items you have, the less you waste.
If you have many items, you'll naturally tend to have less empty months. This effect works two ways, even. For example if you have 0,1,2,3 reservations per quarter you have a total of 30 rows with 3 empty, that's 10% wasted. But if you have 0,0,1,2 reservations you have 21 rows with 6 empty; 29% waste. Even though your content shrank by only half, your waste almost increased threefold.
Even if you don't get more empty months (0,1,1,1) you will still have 21 rows with 3 empty and that's a waste of 14%; higher than the initial 10.
Knowing, predicting, searching, and scrolling.
Knowing where something is is the fastest.
Predicting is slower because you have to go where you think it might be, then have to check, and readjust.
Searching is the slowest, because you have to check each item before you move to the next.
Scrolling with a mouse in any of those 3 is slower than using your eyes to skimming in one of the 3 ways because your eyes need time to re-ground themselves every time the screen moves.
Balancing two scales.
The biggest tradeoff you have to make is about reading versus scrolling.
If you have only a few items and if several fit on screen, people will skim/read the labels until they see the one they need. If something is off-screen they need to scroll to it.
Because reservation get added and removed constantly so you can't know nor predict where they will be within the month. If we have Andrew, Bart and Charlie, and Andrew cancels, the other two will change their position within the month. Same if we have reservations on 1st, 12th and 23rd, and 1st cancels.
It's worth dropping some empty months to reduce how often you scroll. You incur a slight slowdown in the single-screen navigation by going from predicting/counting the position of a month to having to skim each month label, but it outweighs scrolling.
But if you have a lot of items, you are going to have to scroll. We can't know where a month starts because the size of them fluctuates depending on amount of reservations, but we can predict where they will be because the months themselves follow a pattern. So users can count while scrolling in one motion, and predict 'February, March, April' rather than having to read the labels. As predicting is faster than searching, we want to let users predict where they are, which means leaving the labels in.
Another way to look at it is minimizing the amount of action changes. If you can keep them reading labels quickly without scrolling, that's perfect. If you can let them scroll without having to read, that's also good. But you don't want them to scroll read scroll read etcetera.
How important is date-searching?
A lot of factors go in here, so I won't go in detail but let's compare your content vs a generic blog. Type of content; reservations are highly time sensitive, blog posts aren't. Recognizability; reservations all have name/date/room, blog titles are designed to stand out from eachother. Need for specific information; if I want to read up on some CSS techniques there are many interesting related articles. If James Brown checks in, I have zero use for reservations for James Dean or Charlie Brown.
So in short, your user values date-searching a lot.
This specific case
You mention roughly 45 reservations in 6 months. Let's say that's 0,3,6,9,12,15 reservations per month. You end up with 51 rows of which 3 are empty - about 6.5%. So you could save 6% in time not reading the headers.
Based on your screenshots your headers and items are about 50 pixels high, so the whole page would be 2550 pixels. Realistically you have something like 600 pixels per scroll motion (1080p screen with browser tabs, task bar, app menu/header, some anchoring text), and then you skim and reground. That's 4 scroll actions for the whole list.
If you let people predict where they are going to end up (by making each month easily countable, using a good header and using different background colors per month) you can take this down to 1.5 scrolling action while counting blocks. 1 scroll where you predict you need to be, then skimming the page, and sometimes you're on the right bit, sometimes you need to scroll up or down once more.
So yeah, I'd definitely add all the months in your case.
But beyond that, think of other ways to optimize the navigation. How about a dropdown or list with the months at the top of the page? Then they only need to skim 6 lines of text and click one and they'll guaranteed end up at the right month. You could even combine it with a sticky header (https://codepen.io/chrissp26/pen/gBrdo) where the header is a list initially, and becomes a dropdown on scrolling.
