While space is an obvious part of the equation, it's not the main one, you could simply have a sliding physical keyboard just as previous generations of smartphones and be a happy camper. However, physical keyboards had several issues:
- smaller keys than on-screen keys
- structural weakness
- short lifetime (the flex connector and pieces of sliding keyboards had a very short duration span, usually around 18 months)
On the other hand, on-screen keyboards have bigger keys, they don't add an additional component that can break and its lifetime span will be as long as you can take care of the phone, or technological advances make it obsolete.
However, as said, this is not the main reason, just an important one to justify something way more important:
The design philosophy used for smartphones and touch screen devices is that the user needs to jump between dimensions as little as possible. A physical phone is a dimension, everything inside it is another.
Let's say: a mechanical and a logical/software dimension. The idea is that users are immersed inside this dimension. You can read Apple design principles in general, but you might be interested in this part at iOS Human Interface Guidelines : Design Principles:
When people directly manipulate onscreen objects instead of using
separate controls to manipulate them, they're more engaged with their
task and it’s easier for them to understand the results of their
Using the Multi-Touch interface, people can pinch to directly expand
or contract an image or content area. And in a game, players move and
interact directly with onscreen objects—for example, a game might
display a combination lock that users can spin to open.
In an iOS app, people experience direct manipulation when they:
Rotate or otherwise move the device to affect onscreen objects
Use gestures to manipulate onscreen objects
Can see that their actions have immediate, visible results
Additionally, this philosophy is related to a marketing plan where each additional feature, peripheral or physical device that is not strictly necessary for the basic functioning of the main device, is not included by default. Thus, if you want a keyboard, you can buy it. If you want a DVD recorder for your laptop, go and buy it. If you want a productivity app that always came by default, now you have to buy it. This is a huge marketing move for Apple which was quite criticized, yet they're going deep into this move and there are no signs they plan to give up any time soon
This decision is based on a philosophy rather than a technical choice, and it has been highly studied and tested