We presented a genuine client
case study to a group of community professionals.
The challenge was to diagnose the health of the community.
We provided figures, data, full access to the same resources we had and very
broad guidance. The guidance was broadly to track the number of active members,
track the posts per active members, and see what’s happening.
There was a catch (that most missed). The platform the
client was using didn’t track any of the above.
Very often, a client community
is a puzzle with pieces missing.
Most people fell into one of
four groups.
Group 1 didn’t bother to reply to the challenge. Presumably
they lost interest in the job or were overwhelmed by the challenge.
Group 2 responded to say it’s not possible without more
data. One participant’s dad called to complain the challenge was too difficult and we were violating US
employment law. This wouldn’t help our clients much.
Group 3 tried and failed. They used the easy, available,
metrics to get the wrong answers. For example, they divided the total number of
posts by the total number of registered members. This presents a figure that
will usually decline forever (can you guess why?).
Group 4 (around 1% to 3%) made a reasonable attempt at this.
First, they identified relevant proxy
figures.
For the number of active members within 30 days they
typically used Google Analytics to review the number of unique visitors to a
member’s only page of the community within the previous 30 days.
Then they used a large random sample of these active members
to track the number of contributions (this is a LOT of work) or, more simply,
divided the number of posts that month by the number of active members that
month.
A few even realized you could list members by the number of
contributions they had made and made a reasonable effort at removing the top
and bottom 10% to exclude the outliers (members that skew the stats by posting
100 times per day).
The Results
If you do this properly you will get data that looks similar
to the graph below. The number of active members is increasingly as is the total
number of posts in the community. This would make the community appear healthy.
However, by tracking the number of contributions per active
member we clearly see this community is heading for disaster (as seen below –
posts per active member on the right).
If we had followed the data provided by group 3 (that used
the wrong metrics) we might conclude the community was healthy and growing at a
steady rate. We wouldn’t identify the problem here (declining interest, reduced
social bonds, poor conversion, rapid growth reducing social ties) and takes
steps to resolve it.
This is the art in community science, constructing reliable
models to get valuable data to diagnose and improve the health of a community.
As we’ve seen, very, very, few
community professionals can do this.
Summary
Groups 1 and 2 didn’t bother to try. They had absolutely no
grounding in how to analyze communities. This is the group that’s flooding the
community space at the moment. This group’s approach to communities can be
summarized under be nice to people on the
internet. They don’t appreciate the value of the science-driven approach.
Group 3 appreciates the value but not the methods. Reading
books on this topic would help, as would constructing models to measure your
community that go beyond the data Google Analytics gives you (e-mail me if you
want my feedback on your model – I’m eager to help).
Group 4 are the most valuable people in the community space
(so valuable we recently launched a community just for this group). This group
can effectively analyze a community using data and develop theory to identify
what to do next.
Sorry for the long post, but this is important. If you’re in
the community profession, go beyond being nice to people online. Focus your
efforts on getting good data, mastering theory, and effectively putting this
into action.
There are thousands of community professionals that can be
nice to people online and a mere few community scientists.
Work for FeverBee: We're hiring a community manager and a community coordinator. If you're a community manager that loves the data/theory behind communities, click here. If you're new to the community profession and want to work with us, click here.
