Would inviting existing users hurt the results of my summative
usability test too much?
Should I rather invite existing users for a separate test?
Is there a benefit of including existing users?
Think about this: SUT is designed to be ran on laboratory conditions, with a very specific set of guidelines oriented to define usability benchmarking, which is defined after rigurous statistical analysis. If you mix your audience between experienced and non-experience users, you'll certainly have a contaminated sample, because while not impossible, it will be very difficult to distinguish these users. Furthermore, running ANOVA will give you really unpredictable results. If you REALLY need to do it, better go with chi-square (and make sure you have a dimension to recognize experienced and novice users) or factorial ANOVA (take a look to an example of the use of factorial ANOVA on an HCI test pretty similar to yours, specially when it comes to the two ways you'll need to define), but it will always be more accurate and easy if you run 2 separate tests