Measuring An Online Community: Master your data to gain an unfair advantage

You have a truly remarkable advantage over offline community builders. You can track every single action your audience makes. You should know exactly what stage they are at in the membership life cycle process and which stages need to be optimized.

I'm always amazed by the number of organizations and community managers which have either a) No strategy for the community beyond maintenance or b) A strategy built upon guesswork and assumptions when the data is so close at hand.

You shouldn't be guessing what is or isn't working in a community. You should be religiously gathering and analyzing what the data. You should measure the following:

• New visitors. This shows whether your outreach is successful. Always compare it to the previous month and six months ago. You should also analyze where these visitors arrived from and track how many of each progressed into active members. You can also track the success of each different source of members (where does the best quality traffic come from?)


• New visitors to new registered members. This shows whether your website is optimized for converting a curious visitor into a member and whether you're attracting the right sort of visitors. You can go further and measure their progress through each stage of the registration form.


• % members which make a contribution. This shows whether you are converting those that register into participants within the community. If this is low, you might be just collecting lurkers. 


• Members active within the past 30 days. This shows whether you are gaining or losing active members. When this number starts to drop, you have a serious problem and a limited amount of time to correct course. 


• Contributions per active member per month. This is an activity per member ratio. If this drops, members are less engaged in the community and could lead to more members leaving. This might also show if a small number of members are dominating the discussions. 


• Visits per active member per month. This shows how often members visit the community. The less frequently members visit, the more likely the contributions will drop and the number of active members will depart. This may also show the popularity of events held in the community.


• Content popularity. Each piece of content can and should be measured. How many people read it, how many responded to it. This will indicate which content items are most popular and which should be discontinued. 

 

You should also use sampling to understand the following:

• What % of newcomers remain members for more than a month. Select 10 newcomers from three months ago and analyze their journey through the community and specifically where they dropped out of the process. Did they make a contribution? Did they not make a second contribution? You can adjust and tweak your community for this. 


• Speed of replies to discussions. How quickly are discussions receiving a reply? The faster the responses, the higher the level of social presence within the community and the greater the level of participation. 


• The % of newcomers which initiate a discussion. This highlights whether newcomers may be unmotivated or intimidated to start discussions. 


• Language and tone of voice. What language do members adopt when they address each other? Is it formal and polite? Is there friendly banter? Is there a sense of familiarity? This will let you know what stage the community is in.


• Sense of community. Ask members every year to participate in your specially modified version of the sense of community index. 


• Number of volunteers. This will indicate the number of people moving on to the highest levels of engagement within the community. Low numbers usually limit the scability of the community.

Each piece of data will tell a story. If the number of active members is decreasing but the level of contributions continues to rise, it might indicate a core group is dominating discussions and other members are unable to break into the circle. As a result you might provide core members with a separate place to chat, or work to break newcomers into the group or talk directly to group members about the problem. 

Create a spreadsheet and a graph showing all this data. Update this monthly. Watch for numbers that dip and take a corrective course of action. 

When you gather data you can set objectives, strategy and targets for each of the areas of community management (growth, moderation, relationships, activities, content etc...). 

In practice, if you notice the number of volunteers has dropped, you can set a relationships strategy to focus on fewer bring and offer opportunities to be involved in areas of the community they are passionate about. 

Additional Resources

• The Lithium Community Health Index. A complicated measurement to provide a single number for the health of the community. 


• The ROI of online communities. A simple method for organizations to understand the benefits they receive from their online community. 

Comments

[Stephen]

Such a great post! Agree 100%! Thanks for sharing.

[Christinegeraci]

Agreed, this is a great post. Do you have any thoughts on ideal benchmarks for each data set over time, or do you think it's all just relative to the type of community you manage?

[Rob Elliott]

Great stuff, this is one of your most interesting posts recently. I think we'll try to do these measurements on the Wikidot community.

[Wheelygrl]

Thanks for this. So many good thoughts here I can put into practice right away!

[Dan Limbach]

Excellent post. Which tool (or tools) do you recommend for compiling and analyzing the objective data?

[Mike Goncalves]

Terrific post Rich. This certainly helps identify the important areas to focus on and analyze when running a website. Thanks!

-MikeG

[Midnighthaircut]

This is a great post. I completely agree that the opportunities to make informed decisions (rather than intuited ones) is unprecedented.

But we also have to keep in mind that it's the analysis that's really valuable, and the data is just a tool to get there. Something like 'Members active within the past 30 days' is great as a general benchmark of community health. But it won't tell you why the numbers drop when they do or climb when they do, which is what you need to know in order to make tactical decisions.

Measuring active monthly users will never tell you what your community needs. The right measurement program combined with rigorous A/B testing is key to gleaning insights into what tactics nurture an engaged community (and what tactics drive members away).

[Lalitha Brahma]

Great information. It is tough to keep track of all the above, but having this knowledge and making a commitment to improve in each area, will help increase ROI. Thanks Rich.

[40deuce]

Great advice in here, but I have to agree with Mike above me. Numbers are great, but when it comes to community it's about people and people require quantitative analysis.
The numbers are a great place to start and I'm sure that most people that deal with online communities will be able to use at least one piece of advice you've given here, but don't forget the analysis part of both those numbers and your community members.

Cheers,
Sheldon, community manager for Sysomos

[MPGodfrey]

Great post. Data-driven insights are what bring brands and their communities close. The analysis piece is the driving force behind the intelligence needed to connect people with content that is customized for their consumption, and the driving force behind engaging new contributors and points of view. Looking forward to reading more from you.

[Kadee]

This post is very helpful, thank you for sharing. Is this is a measurement plan for websites in general or are you speaking to social media communities? If so, how can you know where your visitors are coming from? Thank you.