I came across a few articles (e.g. here and here) with a list of "usability testing as a service" providers, and was wondering if it's so great, why doesn't everyone use it? Is there a clear list of pros and cons for using such services versus "traditional" methods?
1 Answer
It's pretty much and impossible question to answer - the list of tools you link to are very different and best used in quite different contexts.
For example Silverback is a great tool for folk doing on-site usability tests. If you've using a usability lab - probably a waste of time. Google Website Optimizer - great for cheap A/B testing. Not useful for anything else.
The above two tools aren't comparable in any useful way - and the same applies to much of the rest of the list ;-) Use the right tool in the right context. You might find the Remote Research book a good read http://rosenfeldmedia.com/books/remote-research/.
I guess the most general observation I could make is that there's a big difference in the kind of feedback you get from doing in-person vs remote testing, and even more if it's outsourced to a third party. The more you need good quality qualitative observational data - the more likely traditional in-person usability tests are going to be more useful to you.