The default behavior from a user will be that they just try to touch anything that looks like it could react to touch. Your mockup only shows the numbers in the first place, so chances are that the first thing users will try is to tap directly on them.
Generally, this is a problem the OS designers also face, so a look at their solutions provides further insight:
The key is, as has already been mentioned, that your design is lacking affordance to slide. In my opinion, it lacks any affordance at all – the user will probably still try to touch the numbers, but that's not because they afford it. It's because there's nothing else they can try.
However, the numbers don't afford sliding. You're going for a split-flap display look – now in theory users could make the connection that these are, mechanically speaking, plates rotating on an axis. On the other hand I'm not so sure how many of today's smartphone users have ever owned a device with a mechanical split-flap display, or have seen any "real world" example of it at all.
So, going by the examples from the OSs you could try to …
Another, more general question: Why is the touch-sensitive area limited to the grey area? Are there other UI-elements above and below it? If so, they may draw users' attention away from the numbers. If not, why is there "dead" area instead of everything above the numbers increasing the value, and everything below them decreasing it?