My office has recently created a bunch of computer based trainings (CBTs). They're basically slides with audio. Some of them automatically advance to the next slide and others require a manual click for each slide. Does the degree of interactivity with such a presentation have any effect, positive or negative, on how well the user retains information and thus performs on the evaluation?
I think the goal of a training is to engage the trainee. This could be done through interactivity, but please avoid making interactivity the goal. Often we talk about such effects in UX and psychology, but a perfectly designed interaction pattern can never compensate for bad quality content.
Some talks (pure video, for example, TED talks / Coursera) can be much more engaging than interactive training material.
Strictly answering your question: interactivity can be a good way of engaging your audience, for educational purposes. Asking them questions (even without expecting answers) will already prime them. Forcing your users to press a button to advance to the next slide may have an effect, though I doubt it is significant.
I think a stronger reason to switch to manually advancing through the slides, is that auto-advancing takes away user-control and might interfere with the training itself: perhaps the trainee did not yet finish reading/understanding the current slide?
Yes. Interactivity is critical for learning repetition and re-enforcement. ARS (Automated Response Systems) such as the multiple choice clickers used in universities, continuing medical education, etc... improve learning. Also take a look at Kahn Academy
They demonstrate very well across their learning paths, and activities how interactivity enhances the learning experience.
Another great example is the Redis (nosql db system) tutorial interface: