Both usability and A/B testing are tools at your disposable. Both are important and should be used as part of your larger toolset, which includes other things such as heuristic evaluation, market research, etc.
Usability will, to some extent, always be suppositional or clouded or skewed; whatever term is appropriate. There's often a lot of set-up and explanation associated with usability, which is why we write test backgrounds...to bring the participant up-to-speed on the scenario and the test. Then we say things like "be candid in your feedback" and "you won't hurt my feelings".
By trying to set the context of the test with the user, you're already asking them to imagine a scenario and they will begin to project attitudes and responses, based on what they imagine. And then there can be bugs in the prototype. Because of these issues, your findings may be directionally accurate but will never match a true user in their true environment in your production app.
Production testing is the only way to truly measure the effect of the product. A/B testing is for fine tuning the effect of a nuance of a product. A/B and multi-variate testing can be used for lots of reasons, so you'd be best off researching the correct test type.