While it seems that the general consensus on custom scrollbars is that they are somewhat detrimental to overall user experience, powers beyond my control are suggesting their implementation is mandatory. Other than giving the scrollbar standard accessibly features (wide enough to be easily selected, high enough contrast to be easily noticed, etc), what are usability features I should keep in mind for an easily used custom scrollbar?
Good old Jacob Nielsen on Scrolling and Scrollbars:
With each of these successful designs, users easily noticed the scrollbars and used them without problems. By comparing the two figures and several other designs we tested, we determined four requirements for a good scrollbar:
- Use an actual bar in the shape of a rectangular trough, preferably in a color that contrasts with the background.
- Show arrows at the top and bottom.
- Include a slider (sometimes called a "thumb" or an "elevator"), preferably in a color that contrasts with the trough. The slider's position should show the visible area's position relative to the total area, so people can see how much more content they have to scroll through.
- Allow users to scroll by
- clicking in the trough,
- clicking on the arrows,
- dragging the slider, and by
- using a scrollwheel on their mouse.
Scrollbars are easy to get right. In fact, the less work you do, the better your scrollbar. Usability is almost always enhanced when you use the built-in scrollbars rather than design your own.
what are usability features I should keep in mind for an easily used custom scrollbar
The big thing is are they usable regardless of the input device? Can I scroll with the keyboard, the mouse, the mouse wheel, by touch? Does it hinder any assistive technology (such as screen reading software)?
In the end, though, as you seem to already realize, this is something that one probably shouldn't do. It's akin to a client asking you to build a custom web browser just for their site. It's silly on one level, arrogant on another. Sounds like you already fought that battle, though.
The important thing is that it behave as much as possible like the standard OS provided scroll bar. As DA01 mentioned they should respond all the normal inputs, including page-up/page-down, arrow keys and space bar. Also scrolling should be done at the same speed that the OS would do it.
One of the biggest reasons to not to do custom scrollbars is because it could easily feel nonstandard on certain platforms, and significant testing is needed to understand the extent of this and know if it is a problem. And properly supporting touch UIs suddenly becomes complicated - you need to detect touch platforms and remove the space wasting custom scroll bars.