Draggable objects on mobile devices are object represented with a handle. In iOS apps it is commonly done similar to the Reminders app, with the icon on the right.

Instead of the "scroll area" you may consider a "drag area", or better yet a drag icon that only activates the drag feature when pressed and held. The user could scroll without concern and they would only drag when they press a drag icon.
Yahoo Weather, for iOS, does something similar to this:

Notice the drag icons in the top-right of each container. The user will scroll the screen if they press and drag anywhere else on the screen. Pressing and dragging on this small space will drag a container elsewhere on the screen.
That's nice, but I often find myself grabbing the drag handles by accident.
The way the Reminders app, and others, address this is to only allow dragging in an "Edit" mode. Notice in the first picture that I can also delete, change color and sharing options; there is a "Done" button at the top-right. That's because it is in an edit mode.
Providing such a mode may allow your users to both scroll and re-order more easily.