If you convert the color to another colorspace, e.g. YIQ, YUV or better yet CIE-L*ab, CIE-L*CH, then instead of RGB's Red Green and Blue channels you end up with three different channels, where one is the intensity.
In YIQ, YUV the Y channel approximates the intensity and in Lab and LCH the L channel does this.
You can then easily reduce the intensity channel, then convert back to RGB.
For high contrast, I recommend changing the intensity until you have at least half a range contrast between the foreground and the background.
E.g. if you range is 0...255 then at least 127 difference in intensity channel.
Note: YIQ has the simplest conversion from and to RGB:
Y = 0.299 * R + 0.587 * G + 0.114 * B
I = 0.596 * R - 0.274 * G - 0.322 * B
Q = 0.212 * R - 0.523 * G + 0.311 * B
R = 1.0 * Y + 0.956 * I + 0.621 * Q
G = 1.0 * Y - 0.272 * I - 0.647 * Q
B = 1.0 * Y - 1.105 * I + 1.702 * Q
Edit: Using the W3C guideline
Contrast ratio: (L1 + 0.05) / (L2 + 0.05)
where L1 is the relative luminance of the lighter of the colors
and L2 is the relative luminance of the darker of the colors.
Contrast (Minimum): For regular text 4.5:1, for large-scale text 3:1
If the background is brighter, then use the Y channel of the background as L1 and the Y channel of the foreground as L2.
If the foreground is brighter, then use the Y channel of the foreground as L1 and the Y channel of the background as L2.
If you are using a 0...255 range for RGB channels, divide them (or Y) by 255 before using in contrast formula, as the formula seems to be for the normalized range of 0...1
The contrast formula demonstrates that the darker the colors, the smaller the difference between the colors needs to be in order to achieve a high contrast. (Bright colors on black are easier to perceive than light colors on white.)