I am in the process of conducting some traditional usability test sessions for my sports website. I have some good feedback so far on the navigation, layout and design of the website.
But I've just read up on guerrilla testing and think it may be useful to get more feedback.
However the two types of usability test means there will be variations in the test proceedings. For example, the guerrilla test sessions would only last 15-20 minutes at most so that would mean I have to reduce the number of activities each participant does. That means I'll have one set of participants that did 10 tasks and one set of participants that did about 3 or 4 tasks.
Would the difference in testing environment, test type, tasks given to participants etc.. cause reliability or inconsistency issues when it comes to reporting results? Would it be suitable to draw conclusions with two sets of test results from 2 studies?
For example in my main usability test I'd ask to register tickets for an event and in the guerrilla test I'd ask the participant to do the same. If both report issues with the navigation, could I still draw conclusions taking consideration the environment and difference in test type?
Should I only stick to one type of evaluation method when gathering data from users?
I guess there isn't a generic answer for this question, but it's more to do with the necessity to conduct both test types, if it may improve or strengthen the quality of my feedback.