The Gmail suggestion is great, but I'd also like to point out how Laravel Nova (and I'm sure lots of other UIs) do this.
Nova shows the "standard" multi-select checkbox at the top of the page, but clicking it drops down two checkboxes. One selects all on the page, the other selects all that match your query. It also shows how many items it'll select when you do so.
The wording could be better (e.g. "Select All on Page / Select All") and it's a bit confusing clicking a checkbox and not having something checked immediately, but it forces you to make a conscious choice about what you want to select.
Gmail's way of doing it is great, but the notification that you haven't selected everything comes after you've made a selection, and personally I prefer to know what I'm selecting as I'm selecting it.
As a side note if you wanted to go with Gmail's way of doing it, I'd suggest making the notification appear in a suitable colour (e.g. yellow) then quickly fade that yellow out to give a visual clue to the user that a new piece of info they should pay attention to has shown up. Here's how Google Chrome's dev tools does it to show that an element in the DOM has changed: