Don't design a UI based on what people tell you they'd like -- that's a core tenet behind design in general. Otherwise, you can end up with convulted, hard-to-use interfaces. User interviews should serve as research into what kind of mental models, goals, needs, and workflows people have, not into feature requests. (I'd recommend you to read Designing for the Digital Age, or some other book or article that covers user research.)
As for your dilemma -- I'd recommend using a button, because:
- Delete is a destructive action. Using a swipe to activate could easily lead to accidental deletes, as swipes are easy to trigger by accident, even if you have an "undo" feature. (This is different from archiving, as that action is non-destructive and reversible.)
- Swipe actions are not discoverable. Even if you teach them when launching the app, the user can easily forget (your app is not the only app they use). Therefore, it would still be best to have a discoverable secondary UI. You can accomplish that with a button, so why bother with an additional swipe gesture?