The technical implementation aspect is off-topic, but I've certainly seen product listings where users can swipe left and right through the product images without tapping through to the product detail page. That sounds like the mobile touch equivalent of the desktop behaviour described in the question. Airbnb's app is one such example:
For non-tech savvy users, it's hard to use gestures to drag items. Random click on the item itself – and user gets to the item's content. Gestures are non-visible, too.
Here is a good point to use modes. Just let a user enters into the safe Change order mode and use buttons for re-ordering. This:
makes it obvious of how to move an item up/down, or make it ...