We are following this pattern, inspired by Mac and Windows OS, but I don't have a solid answer for why web apps use it. What is the importance of the X button? Is it really required?
The X button to close a dialog is probably about the most consistent thing users will see across all dialogs, windows, popups, overlays, etc. It's synonymous with the 'get me out of here' sentiment. I don't even have to read any of the content in the dialog, nor any of the buttons to know that the X button will help me escape without doing something unwanted. I don't have to understand the buttons in relation to the message. I don't have to decode their meaning or determine which is the right one - even if they are well labelled (although many are not). It's pure instinct that as Juliano says, we've learned and inherited from almost every other app we've ever used.
It's not required in the strict sense, but it is beneficial to the user's experience. If the X button is not there, I'm forced to make a decision between which button to choose in order to escape, and that's not good. There are so many instances of bad dialogs, and awkward messages, and ambiguous buttons (especially cancel buttons are shown on a dialog that pops up while cancelling an action).
It's a shortcut, and it's comforting, and it's a basic expectation.
We could talk about mental models Users project their knowledges from another softwares, platforms, with this in mind we understand users sometimes prefer the X button instead the cancel, for many reasons, more faster, they don't need or understand another buttons bar, and will spend more trust.
When we see a notification popup asking for user accept or decline, I heard from many users they prefer close for X button, instead click over Sorry No, because in their mind they imagine, "If I click in button Sorry no, something will happen and I don't wanna this". All of this, is because their project their knowledge.