Making Sense of Data: Is Your Community Healthy?

This question came up recently:

Should you be
concerned if the % of engaged members has fallen from 7.88% in March to 5.03%
in June?

It’s tough to answer this
without more data.

For example, if the community has been growing, there might
be more members participating more frequently. However, the total % of active
members to registered members would be lower. This indicates lower conversion
rates (newcomers to regulars) or higher turnover of existing members (existing
members sticking around for less time). Both are a cause of concern, but hardly
fatal.

However, if the community hasn’t seen a significant increase
in registered members, this shows that members are leaving. This is a major
cause of concern.

This is why we need more than one data point. We need to
track membership, activity, and sense of community.

Membership

First, lets analyze membership. We want to know:

• The
  number of active members within the previous 30 days. This is the key
  metric to highlight whether the community is heading in the right direction.
  The overall number of active members shows how your community is doing. Note we
  want absolute numbers, not percentages.

• The
  number of new registrations. This shows whether the community is still
  attracting new members. If this begins to drop, activity will soon follow. This
  shows how successful your promotional efforts have been. 

• The
  number of newcomers to regulars. We covered this here. Broadly,
  we want to track how many newcomers make their first contribution, and how many
  stick around 6 months later. This shows if you’re wasting potential.

Now you have a story. If the % of engaged members has
declined, you can show it’s either because conversion rates from registered
members into regular participants is lower or existing members are drifting
away (or a combination of the two). Once you identify the problem, you can
tackle it with specific interventions.

Activity

Next we track activity metrics. This includes the following:

• Total
  number of posts. This is the broad metric that shows how successful your
  community is. If it’s going up, that’s usually a good thing. If it’s going
  down, that’s a bad thing. A decline in participating members isn’t so bad if
  the ones you’ve got are more active.

• Number of
  new discussions. This shows whether the community is stuck within a limited
  number of topics or has plenty of fresh ground to cover. If your concept is too
  narrow, you might have very few fresh discussions. This can also indicates that
  members have increasing or decreasing levels of social fear.

• Number of
  responses per discussion. This shows whether the discussions being initiated
  are of interest to members (and whether members are visiting enough to see
  these discussions). 

• Number of
  posts per active member. This shows whether members are more or less
  engaged in the community. Use active member, not registered members to get
  accurate data.

• Visits
  per active member. This and time on site (below) validates the trend
  highlighted above. If members are visiting less frequently, that typically
  indicates a major cause of concern.  

• Time on
  site per active member. This shows whether members are finding something of
  interest when they do visit the community. 
  Shorter time spent per vision doesn’t matter, but total amount of time
  on the site does.

Ideally, you want to remove the top and bottom 15
percentile, but this is rarely possible.

These metrics paint a vivid picture about whether your
existing, active members are more or less engaged in the community. They
specifically highlight anything that might be a cause for concern.

If you want to answer questions about how your community is
doing, begin by getting this data.