In general my opinion is to not break the current pattern unless necessary. Consistency is always key. Therefore I think that if you're overriding the default scroll behavior of just panning up and down that there should be some serious thought behind it and usability testing to back it up. My thoughts would be if you're panning content in some direction when the user scrolls, then you're probably close enough where the user won't be incredibly confused and put-off. In general most people will assume it does what it usually does and that it will simply pan the screen vertically when you scroll.
However, I believe the cases you mentioned deserve a bit of the spotlight as well. As far as 3D navigation goes, I would say the Google Maps' override of the default scroll behavior is fairly standard as far as 3D navigation goes. I'm not sure if it's becoming a standard because of Google Maps, but it's definitely becoming commonplace. I do agree it is odd to have two differing patterns of scrolling in the same context (a single web page) though. Breaking consistency is more confusing than anything, which is why this is such a dangerous road to travel down.
As for the QuickTime example, I feel that is staying fairly true to the underlying concept of panning content, although when you think about it, it does feel odd. I believe there are more common interaction patterns for achieving fast forward and rewind a computer, but the granularity of control you get from a scrolling control is much better.
What it boils all down to is whether or not your target audience will find it intuitive enough to discover without effort, have it behave as expected with limited learning required, and that you have a consistent behavior across your application.