I'm working on a usability test of a e-commerce website. The test is a validation test (the kind of test performed close to service's release).
I read that, for these kind of tests, I have to use benchmarking to compare tests' metrics. My doubt is, what is kind of benchmark I must use ?
Would timing be one of them ? if so, do I have to use "thinking aloud" protocol? I saw that think aloud slows down users in performing given tasks, and it "forces" users to think about what they are doing (not properly a typical way of use a system, especially in a environment where there are some distractions).
What do you think about it?
What kind of benchmark metrics I must consider?
In addition, another doubt is, benchmarking compares data gathered during tests with same data gathered before test. My doubt is, benchmarking data must be taken from expert users, myself, novice users or someone else?how can I rely on these data?I don't want to fall in error, considering a good completion time (for example) one that could depend on level of expertise.