16

I am working on an early stage product. We want to understand if there are big shifts in user behavior when the platform has a thousand users vs. in the future when we may have hundreds of thousands of users?

We have a community aspect to the product where people give each other feedback. Right now we are seeing great response rates and high quality content within the community. We currently rarely moderate people's posts.

We also have learning assets within our platform that users can collaborate on. These assets were slow to take off, but are now seeing some usage. We want to increase engagement here without taking away community quality/response rate.

How can we maintain engagement and quality in the community as we grow and increase engagement with our learning assets? How does people's behavior change as there are more users/more activity? How do we keep it relevant for that user?

  • IME, quality and quantity have an inverse relationship in community apps. – plainclothes Sep 11 '15 at 22:29
19

Absolutely

Let's start with just one observation from network theory. Let's say N is the number of people the community. Then the number of possible relationships between 2 users in a community is proportional to N-squared (more precisely, N(N-1)/2).

  • So for a 1,000 person community there are ~500,000 possible relationships between people. For a 100,000 person community, there are ~5 billion possible relationships.
  • That is to say, growing a community by 100x actually increases the density of possible relationships by 10,000x.

In reality this doesn't happen since users don't actually have the time or interest to form a relationship with everyone else in a community, but the N-squared proportionality gives rise to some dramatic dynamics so it's a helpful model to consider.

What are some possible effects?

  • Terrorist effect - All it takes is one person to ruin your day. If users are able to post content or comments publically, scaling the user base can dramatically increase the incidence of nastiness on a site. For example, let's say 1 out of every 1000 is a troll, and posts nasty content once a year. With 1,000 users your community has to deal with a public troll on average once a year. With 100,000 users you have to deal with a troll once every 3 days.

  • Content inundation - As the user base grows, user attention budget doesn't grow. So if an average user posts new content once every 10 days and has enough time to digest a maximum 100 pieces of content a day then:

    • For a 1,000 user community, there are 100 pieces of content produced every day, which exactly fits the attention budget for each user. i.e. supply of content meets capacity to digest content.
    • For a 100,000 user community, there are 10,000 pieces of content produced very day, which is 100x the amount of content users can digest in a day. So now the dynamics change a lot....concepts like curation, crowdsourcing, filtering, and search become important to ensure a balance between supply and demand for content.
  • Accretion effects - User bases rarely grow linearly. They tend to grow exponentially. This has some important effects because the earliest users will become very different from the newest users. Some examples:

    • In Yelp, a brand new restaurant with no reviews has a hard time competing with a 5-star restaurant with 1,000 reviews. The restaurants may be of identical quality, but one has a tremendous advantage of seniority.
    • In StackExchange, the earliest users who answered the simplest and most common questions about Javascript have a tremendous advantage over new users who are trying to answer less common questions.
    • On Twitter, senior musicians who have accumulated huge follower bases have a tremendous advantage over new musicians who start with only a handful of followers.
  • Heterogeneity effects - Early user bases tend to follow more narrow cultural behaviors (e.g. because they are from the founder's locale, age group, friends/family, etc). As user bases reach tipping point and expand rapidly, the demographics often broaden (standard deviation may or may not stay the same or even narrow, but range broadens with N). This can change the complexion of a community, e.g. an environmental awareness site may start in San Francisco where the issues are easy to discuss, but as the community grows may need to deal wtih language, culture, and political issues in India, Japan, Germany, etc.

  • Herd bias - For platforms that use ratings or show views/votes/shares/likes, this is a well known bias. For example, if 10,000 people immediately condemn a suspected murderer online, the next user who opens his news feed is more likely to also condemn the murderer rather than read an article and form his own unbiased opinion.

Any of these dynamics may follow linear, quadratic, or exponential dynamics as a site grows so a lot of social networking site growth concerns itself with adopting moderation, rules, curation, filtering, search, and normative/equalizing dynamics in order to counteract negative network effects and support positive effects.

This is only a partial list of scaling dynamics with communities, but is hopefully enough to illustrate that behavior can vary quite dramatically as sites scale.

  • This is an excellent post, but it would be even better if it could link to external research. We wouldn't want people to think that you just made up convincing-sounding arguments. – Thunderforge Sep 21 '15 at 20:35
6

One issue is one that arises here at SE. It's reffered to as the fastest gun in the west issue.

The gist is, if you have many people struggling to get their voice heard the loudest it will result in less quality content being pushed out faster so that it is noticed faster.

If you have 1000 users the 1% rule says there will only be about 10 active contributors, plenty of room for everyone to be heard. However, if you have 100,000 that's roughly 1000 users competing for the same screen time, which surely lead to poorer content being submitted faster. As @plainclothes said "quality and quantity have an inverse relationship ".

0

Rob May of Backupify makes two useful points about early stage products in his blog post The Dark Side of Customer Development and Lean Startups:

"Our first 500 customers gave us VERY different feedback... they were... hard core tech guys... and wanted features that later customers didn’t care about... we listened intently to those first customers.... then... realized our product could have... mass appeal... but we had made architecture decisions that made some of the... features more difficult to implement."

Rob warns about the "...misleading... insistence on surveying all the early users. We did it, and learned a bunch of stuff we already knew. What we really need to know is why... people... don’t sign up... Surveying my existing user base... will never tell me that."

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.