First off, kudos to you for working on accessibility. I don't know if you're doing it out of the kindness of your heart or if it's a requirement.
It'd be nice for a site to be AA compliant by default rather than having to change a setting, but at least you're providing that feature. And to answer your main question, yes, having the feature is enough to satisfy WCAG 1.4.3 Contrast (Minimum)
Some people need much more than the minimum contrast (4.5), and perhaps even more than enhanced contrast (7, WCAG 1.4.6), and while the latter is a AAA criteria, I'm not sure we want our sites to all be AAA by default. Making everything black on white or white on black (maximum contrast) will be helpful to some people but not others. Even if your site had sufficient contrast by default, the color choices might not be good for some people. There's no way to have a default site be good for everyone (but we can try).
But I do agree with @JonW that, ideally, you should try to be AA compliant by default without the user having to change anything. That's just a good user experience.
As far as an "official link" to support my answer, that's tricky. WCAG is a great checklist to assist in making a site accessible but be careful you don't fall in the trap of "checking off boxes" from that list. You can certainly implement all of WCAG AA and say you're accessible, but it could be a horrid experience for the user if you completely ignore UX.
Whether you implement a WCAG criteria directly on your site (such as the default color contrast or font size or keyboard focus) or if you provide a "settings" type dialog where the user can change these settings, they both satisfy the guideline, but I don't think there's any "official" doc that really says that. It's more about the user experience.
The more you can do for the user without them having to fudge the settings, the better.