I have 50 people lined up to beta test our website for a week. Best way to utilise this resource?

Question

Over the last couple of months we built a platform with a similar goal to http://upwork.com. Basically you can sign up as someone looking to hire help for small projects, or you can sign up as a "freelancer" to help people out. We installed Hotjar to get as much insight out of this as possible.

The goal of the website in these early stages is to get as many people actively participating as possible after launch. We will initially try this in a small-ish town as an experiment. So, conversion is important, but the people have to stay on the platform and use it as well.

We made a smaller scale usability in the earlier phases of the project which was very insightful. It was a guided test with 3 people.

Now we have 25 pairs of people signing up either as a "freelancer" or "employer". We have about a week to do some usability testing with these participants. This is only remotely (although I'm sure I could get 5 people to come in for an interview).

I never had this many people available. How do I go about testing the website as efficiently as possible without going crazy myself?

Thanks for your help!

Answer 1

I would continue with your idea of splitting them upto into freelancers and employers 50/50. Build yourself a brief overview of what features you want your freelancers to focus on, normally this would be your key features since they're what attract the majority to the website.

I would build each tester a checklist, there's no point getting people in if they're going to only look at a quater of the features your program offers.

For the amount of testers you have (50), you could have 10 testers Monday, fix problems Tuesday, 10 testers Wednesday, fix the new problems Thursday, final 10 testers Friday.

I would certainly follow this schema, that way as stated above you're not testing a discovered bug over and over. Another way to avoid this would be splitting the beta testers checklists up, so imagine a pair of two testers completing one checklist individually. By having them a day apart that allows you to fix the bugs that are found.

I hope this helps.

Answer 2

My suggestion would be to break up your testers into different sections. Allow enough time (preferable at least a day) in between each section to fix the problems the previous testing section found. That way you can test your fixes instead of continuing to test something that you know is broken.

For the amount of testers you have (50), you could have 10 testers Monday, fix problems Tuesday, 10 testers Wednesday, fix the new problems Thursday, final 10 testers Friday.

Also be sure to schedule enough time in between the testers to make notes about your findings while it's still fresh in your mind. Try not to schedule them back-to-back if at all possible.