I run a lot of guerilla usability tests on my prototypes at my company. Sometimes, after maybe the second or third participant runs into the same issue, I'll think about fixing that issue before testing with the next participant.
If I fix the issue, I'll immediately be able to test another design that could be better. On the other hand, I'm potentially skewing consistency in my results.
Will the usability of the final design be higher if I update the prototype after only a few participants? Or, should I refrain from updating any prototypes / mockups until all 4-6 participants have been tested?
Before posting this, I found "The RITE way to prototype" which loosely describes this process, but doesn't provide any quantitative evidence on the outcomes. I haven't found anything else, and wanted to see what others have done in similar circumstances.