I don't know for certain, but I would say that pausing a playback is the most common interaction a user does to a running video. Therefore mapping the most accessible keyboard button, space, to access that feature seems quite straightforward. I believe this was the line of thought when the first instance of this occurred in either a video or audio playback program. And from there it has pretty much become an unwritten rule.
And FYI, the 'space-to-pause' feature in youtube and other embedded video players is still available outside full screen mode. Thing is though that this requires that the player has to be focused, if the user has interacted with anything outside the video player in the browser the mapping between the video player and the keyboard is lost, and therefore the 'space-to-pause' feature could appear as being unavailable outside full screen mode.