I am looking for standardised techniques or question sets that can be used to test the value, credibility or trustworthiness of a community advice site.

In particular, I'm interested in testing whether users interpret the site as:

  1. having enough rich content to be worth searching for advice on their topic of interest
  2. being specific enough to offer advice that can be acted on, or which will be effective in removing a worry
  3. being populated by members who are sufficiently like themselves to offer advice that is relevant to the participant
  4. being sufficiently community-led to be seen as independent, and not, for example, as pushing a particular product or approach

Does anyone know of any resources which might help to develop a test of these characteristics?

Note: I have an additional constraint, which is that I need to test designs, and not a live site, so there is a limit to the volume of actual content I can expose participants to. However, I am happy to adapt test questions designed for a live site as needed.

  • 2
    I'm struggling to see the UX question in here. You can have an active and credible community with terrible UX, and a dead community with great UX. How would an answer to this affect your design choices?
    – JohnGB
    Apr 18, 2013 at 8:42
  • It's a fair challenge, and the question has been misinterpreted in one of the answers below. What I'm interested in testing is whether, for example, promoting community-generated content on the homepage in one way is more effective at communicating the site's richness or independence than another. In other words, the landing page designs need to provide a signal to visitors on the richness, specificity, affinity and independence of the site. Question is, how to test their effectiveness in doing that...
    – Justin
    Apr 22, 2013 at 14:38

2 Answers 2


It's a very specific question, and will require a unique solution.

It doesn't appear that you've weighted your 4 criteria, and you've stated that you can't actually deploy the entire solution - so I'd focus on PAD testing with specific questions related to each of your 4 criteria.

Because of the limited amount of data provided, results for factors #2, #3 and #4 will not generalize to the final site. Content, not layout, will drive perceptions of Community (#2), Utility (#3) and Independence (#4).

The perception of "richness" though? That's fairly easy to establish through layout, presentation and marketing.


I am not sure about standardized techniques, but here are some suggestions for each of the questions you want to ask and some caveats. Since you are looking at four separate factors, the weighting that you give to each factor might also vary depending on the type of site, the features it provides or some other confounding factors.

  1. Enough rich content - hard to define what enough is, but the metrics you can look at include: average number of answers per question, average rating of answers, average age of question and answers, average rating of users answering questions.

  2. Being specific enough - average number of tags per question, average length of question and answer.

  3. User similarity - I don't know if you can measure this without getting demographic information from users, but the general age, gender, interests, etc is a starting point.

  4. Community-led - Most sites offering advice will have administrators and moderators, and these people will generally lead the community. I think if you look at the number of users that join, and the number that continue to actively participate (e.g. ask and answer questions) it will give you a starting point.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.