Surely these days users are either using touch devices to scroll with gestures or are dragging the thumb of the scrollbar or clicking inside the track. Would it be a bad idea to remove the arrow buttons from my scrollbars? A trend has to start somewhere doesn't it? :p
The arrows on scrollbars are a functional element. If you click on them they move the screen up or down. Clicking on the area between the arrows and the position marker usually moves the screen up or down a page at a time. They are therefore not redundant as nothing else behaves in the same way. Whether or not they are needed is a different issue.
While my experience has been that most people don't use the arrows anymore, there are a few people (usually older users) that do. So whether or not you should leave them out (assuming you have a choice) depends on your user base and their preferred way of scrolling.
Edit: Jeff caught a mistake, which I have now corrected.
Mac OS X scrollbars has no these arrows (at least by default) for a couple of years already. And it seems like most of the users are OK with this.
But, for Mac OS X it's a system-wide change and every (almost) app is affected by it so everything behaves the same way. Actually, I don't see any reason to drop these arrows off (except for the rare design purposes) from your app at least until it will not be done system-wide or by majors (I mean, popular apps). I do believe it's not a big UX problem at all, but keep in mind that changing the behaviour (even slightly) of the standard controls may confuse and harm your users.
You're all forgetting
Various physical conditions can make scrolling or dragging really difficult. Clicking is a comparatively simple action to do. Buttons help.
Unfortunately, there are more of these "older users" than JohnGB things, it's just we usually don't mingle with them.
Recently I met a girl - she is about 27ish, working in the online marketing industry - and she told me in a conversation that she never uses sliding on mobile phones, she needs explicit buttons for everything.
So, it's not just older users. It's everyone except for the 10 percent of the technically inclined we meet.
Whenever possible you should follow the conventions of the operating system's UI. Many people are still using older systems in which scrolling down a small amount is best accomplished by clicking on the scroll bar arrows.
The old fashioned scroll bars with arrows is best for Windows XP, and no scroll bars at all is best for touch UIs.
I would point out another situation where the arrow buttons are useful.
In situations where you can't click and drag, and don't have a scroll-wheel or similar mechanism.
I've occasionally found them nice to have when on a computer with a cranky/difficult mouse (generally a laptop with a really, really bad trackpad).
If you can't click and drag, your only options for scrolling are to click the scroll bar outside of the actual bar, which will move you an entire screen width, or the arrows, which move the screen the typical scroll interval (generally three lines).
You can also use the arrow-keys, but I'm speaking of mouse-based scrolling only
They are also occasionally useful when you're using something like RDP or VNC over a slow connection. You can predict the amount of scrolling fairly precisely, so you can scroll without needing visual feedback. This is nice when your connection is slow to update the framebuffer.