It would not be inappropriate to remove a dedicated fast forward (FF) or rewind (RW) button from a digital video UI.
What are they designed for?
FF and RW buttons were designed to move quickly through physical tape media. In the case of the image below a cassette tape, but they showed up on media before this.

There was no practical way to move forward or back through the media otherwise, unless you were playing the tape. The functions were meaningful and practical for the medium.
Duplication of functionality.
Duplication of functionality is not always bad, but it is only really good when necessary - safety concerns being a good example. Digital video players almost always show a progress bar - which also tells you how long the video in full is.
Take YouTube's interface:

The progress bar (a single UI element) does the work of many things:
- Current time
- Total time
- Fast forward
- Rewind
- Jump to specific location
FF and RW are accomplished through the action of dragging the "current position" widget. Alternatively the user is able to jump to a specific location by clicking somewhere on the progress bar itself. All of these functions are not possible in tape media, where FF/RW buttons make sense, due to physical constraints.
There is no need to add another method to accomplish what the progress bar already does.
Evolution of UX
What has happened is not the "removal" of the FF or RW buttons, but the evolution of them into the progress bar. The digital interface (computer or touch screen) combined with the capabilities of the digital media (total time available, instant skipping, others) has made it unnecessary to move through the media in a sequential way. The progress bar is FF and RW, for the media at hand.
Precedence
Concerns about if users would miss a FF or RW button should quickly dissipate with a fast review of other media players available to them. Take YouTube, pictured above, on the desktop or any of the following examples taken from my iPhone:
YouTube:

Netflix:

Notice the lack of a RW button, but there is a 10-second skip-back button. Ideal for when you are watching something a didn't quite catch that last bit of dialog. Instead of forcing the user to move the progress bar, or guess by hitting a generic RW button, Netflix has wisely added a simply one touch for quickly jumping back.
Video (Apple's default video player):

All have removed the FF and RW buttons in favor of the much more versatile progress bar. It is highly unlikely you're users would have a difficult time transitioning away from using the FF/RW buttons on your app (if they even still do).
Accessibility
This depends some on how you implement it, but generally speaking a progress bar should be no less accessible than dedicated buttons.
All mobile OS's have recommended button sizes. They normally range from 5-9mm, give or take, depending on the purpose of the button. The hit space supplied to the progress bar's "current position" should be at least as big, and could easily be larger since your app likely does not have many other controls floating around that the user might otherwise accidentally hit. The added length of the progress bar also provides additional space for grabbing.
If your concern about accessibility comes from physical constraints of your user population (age, arthritis, etc.) than how you implement the progress bar can also change. One thought, make dragging left/right on the video move the progress bar too - not just dragging on the progress bar.
Dedicated buttons could easily be argued as less accessible as well.
- Dedicated FF/RW buttons have a set speed. I control the speed when I drag the progress bar.
- That set speed might be too fast, or too slow for me.
- How do I stop the FF/RW action with dedicated buttons? If I'm dragging, I just stop dragging.
- Narrowing in on the expect point I want to be can require multiple FF/RW button presses. Dragging a little before is easy to do.