Traditionally, checkboxes have been used to indicate simple on/off choices. Occasionally, they are used to indicate on/partially-on/off or on/off/indeterminate state (the technical term for that being tristate checkboxes), but that's unusual and I do believe a lot of users get tripped up by that. (For a specific example of the latter, try the folder properties dialog in Windows Explorer when some files have the read-only attribute set and some do not.) For example, the "read-only" and "archive" file attributes are not interrelated, and can be toggled independently of each other. Doing it with any mutually-exclusive control would require four controls instead of two: off/off, off/on, on/off, on/on. It gets even worse if you consider the "some of the selection are 'on' and some are 'off'" state: it balloons to nine (3^2) possibilities. Add a third and you have 27 (3^3) controls to keep track of! MS-DOS used four file attributes: archive, read-only, system and hidden. With "some on and some off", that adds up to 81 possible choices (even without, it's still 16 different combinations to keep track of).
Radio buttons, on the other hand, indicate that only a single choice can be made. They also usually indicate that a choice must be made, either by accepting some default value or by making an explicit choice.
On the web, even with few choices, comboboxes or selection lists of some kind that display only the currently selected choice (until the control is activated) seem to be common. I'm not sure what the actual usability aspects are, there, myself, but radio buttons can certainly be made less readable. For any non-trivial number of choices, there are certainly upsides to a combobox approach from a UI design point of view.
Personally, when I see radio buttons I know that only a single choice can be made. If I see checkboxes, I generally expect that more than one choice is possible -- and when the UI starts to actively work against the user's expectations, it causes frustration and acute forehead contact with wooden or other skin-covered surfaces.
As for references, I'm not sure how much it counts but even Wikipedia's introductory sentence on the subject says that "a checkbox (check box, tickbox, or tick box) is a graphical user interface element (widget) that permits the user to make multiple selections from a number of options (contrary to a radio button where only one choice is possible) or to have the user answer yes (checked) or no (not checked) on a simple yes/no question.".