Jakob Nielsen's 10 Usability Heuristics for User Interface Design has two points I want to focus on that shows that not having a cancel - or emergency exit - button is a bad idea for usability.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
from Jakob Nielsen's 10 Usability Heuristics
The alternative is to rely on an undo feature, but the fact that a user can delete (and will) information by mistake should be prevented in the first place.
The Usability Body of Knowledge lists these three principles of usable design under error prevention:
Error Prevention and Handling:
Forgiveness: Allow reasonable variations in input. Prevent the user from making serious errors whenever possible, and ask for user confirmation before allowing a potentially destructive action.
Error recovery: Provide clear, plain-language messages to describe the problem and suggest a solution to help users recover from any errors.
Undo and redo: Provide "emergency exits" to allow users to abandon an unwanted action. The ability to reverse actions relieves anxiety and encourages user exploration of unfamiliar options.
from Usability Body of Knowledge: Principals of Usable Design
So clearly there are substantial reasons to not remove the emergency exit paths as user will make mistakes.