There are a few reasons that come to mind, but in my opinion, there is only one that matters.
The automatic transmission has used pretty much the same interface since it was invented and mass-produced in the 1950s. The primary variation over its history is column shift vs. floor shift. This means it is familiar to pretty much everyone who is old enough to drive, including the elderly who remember when the automatic transmission was a new thing. This interface is very stable and familiar to billions of people. There is a strong argument that such a familiar and stable interface should only change if there is a really good reason for it.
While there may be newer, "better" (subjective) interfaces, manufacturers must weigh the perceived improvements against potential confusion among customers. Confusion that, with one design implemented by Chrysler, has resulted in very real consequences involving accidental property damage and even death. All because a manufacturer decided to tinker with a decades-old interface in a way that broke the feedback loop between driver and transmission.
Keeping with a design such that any adult can sit in the driver's seat of an unfamiliar car and be able to drive it with minimal adjustments is a very real benefit. Imagine borrowing a car from a friend or rental company and needing to relearn how to operate the car. That could be an unpleasant experience if you are in an emergency or other urgent situation. Who cares if the dashboard has better space efficiency if you have trouble getting the car moving?
Yes, the traditional gear selector is less space-efficient than newer designs. However, it is critical to provide proper feedback to the driver regarding the status of the transmission. Using the "old" interface appears to be an acceptable compromise for the ability to have both tactile and visual feedback about the transmission in a split second.