Three Very Different Ways To Analyse An Online Community

Most people work from a simple assumption (e.g. “higher levels of activity per member, an increase in retention rates”). This means you measure activity per member and design your engagement activity to maximise activity per member.

The downside is you haven’t proved if the relationship is true (i.e. does it correlate well?), nor the influence levels of activity have on an individual’s retention rate (does it account for 100% or 5% of retention rate increase?), nor whether the relationship is linear (does it drop off after a certain level?).

You could blindly be pursuing more activity when data might show you that 3 posts per member, per month, is enough.

A better approach is to test a falsifiable hypothesis (e.g. sample members by levels of activity and compare this with customer retention rates to prove if the relationship exists and how influential the level of activity is in that relationship). You could then focus on increasing the level of activity from a specific segment to see if retention rate among that segment rises (this isn’t a natural experiment, but it’s still good). You might find that there is a relationship between increased levels of activity and retention, it’s nonlinear. After 5 posts per month, there is little impact.

Now instead of trying to get every member super active, you focus on ensuring they make 5 good contributions per month. This changes how you work a little.

An even better approach is to run a regression analysis to identify which variables correlate with increased levels of member retention. You might find that increased activity accounts for a 27% increase in higher retention rates. This also highlights other key variables. You might find direct messages between members have a 25% influence, opening newsletters have an 18% influence, and adding a profile picture have 10%. You can now test these relationships and build a mechanistic model.

Now instead of trying to maximise member activity at all costs, you might spend more time on newsletter subject lines and content, persuading members to add their profile picture, and ensuring members befriend each other to increase the number of direct messages.

This isn’t easy to do, but that’s exactly what makes it valuable. You can build a specific, tested, model that will show you exactly what you need to spend your time on to achieve. This lies beyond the endless hunt for more engagement. It’s where you can take your work to a more advanced, strategic, level.

Comments

  1. Ernesto Izquierdo says:

    Hi Rich,

    I find that your post is also in line with the benchmark results from activity and social cohesion made the team of Laurence Lock Lee. His results show that it can be actually dangerous to only focus on activity if what you are trying to get is valuable collaboration.

    "Using data from more than 135,000 people across more than 20 organizations, we have uncovered the following relevant key insights:

    1. There is no association between activity levels and response rate. We expected that the more activity a network has, the more likely it is that posts are replied to. Getting a response to a post is obviously essential for collaboration — otherwise you are talking to yourself. So it was a big surprise for us to discover that there was no association between activity levels and a response.
    2. Social Cohesion (measured as the intensity of reciprocated interactions) is the single measure that most differentiates the organizations in our benchmarking sample. Measuring social cohesion is equivalent to measuring your blood pressure — it is an essential health indicator. There is no association between activity levels and social cohesion. We found examples of highly active networks with poor social cohesion. That’s like having lots of people at a party, but the dance-floor is empty.
    3. After the initial launch hype, the average adoption rate is around 24 percent with the highest being 63 percent. C-Suite executives are looking for a fully inclusive collaboration solution. Adoption rates are still a major challenge."

    The lessons are clear, we should create our engagement strategies based on what we want to achieve, not just generating more activity on the community or network.

  2. Sarah Hawk says:

    Any idea how to measure social cohesion?

  3. Ernesto Izquierdo says:

    I've got some hints from that article, but I think that we have two options:
    - try to figure out with our teams (re-invent the wheel)
    - Or ask Laurence to get the specific way they do it. I would imagine that he might charge for that service, but since we don't have Yammer his solution won't work for us. And it will might still be easier for us than trying to reinvent the process.

    Here are the hints I talked about written in his interview by Virpi Oinonen:

    "Our ‘Social Cohesion’ measure is a simple proxy for ‘reciprocity’. It simply measures the degree to which your interactions are reciprocated by others in your network e.g. you reply to my posts and I reply to yours. Aggregating individual results to the whole enterprise means that we have the ability to assess social cohesion at all levels of an organisation. In network science, reciprocity is a fundamental and key indicator of trust and therefore performance."

  4. Richard Millington says:

    I'm not sure social cohesion (by that metric) is the best metric here.

    Let's imagine I ask a question in an internal community and someone with
    the right expertise answers the question.

    By the social cohesion measure, I should then answer any question they then
    ask.

    The problem is I might not know the answer. In fact, statistically, the
    odds of me having the specific answer to a question from someone that
    answered my question must be extraordinarily low. It feels like this is
    measuring the wrong thing. It doesn't matter if the same person answers my
    question, only that my question receives a good, quick, answer.

    I think better measures here aren't social cohesion by that definition, but
    I'd suspect more along the lines of speed, quantity, and quality of
    answers. An internal community has to be seen as more valuable than any
    other communication method.

    • Rich
  5. Ernesto Izquierdo says:

    Thanks for the replies. For the Q&A section, we are focusing indeed in quite specific metrics:

    Questions asked, solved.
    Existing questions (and their answers) appear as you type.

    Concerning efficiency:
    - % of new questions that get an answer within 2h (we are aiming at 50%)
    - % of new questions that get an asnwer within 24h ( we are aiming at 70%)
    - % of questions solved (we are aiming at 90%)

    Concerning connections:
    - % of questions solved by people from other locations (our objective to connect people from across regions)
    - % of questions solved by people with higher or lower positions (our objective to connect people from across regions)

    The social Q&A tool we are trying is called Starmind and it brings the statistics in its service, the goals we fix are the ones that they obtain with other networks with over 5K users.

    I know that Philips gets 50% of new questions answered within 1h, but they have 200K employees.

    I would think that the social cohesion could be more interesting with regards to blog posts, micro-blogging, or comments on documents.

Make your own comment on this post at FeverBee Experts

©2017 FeverBee Limited, 1314 New Providence Wharf, London, United Kingdom E14 9PJ FEVERBEE

We’d like you to join us.

We’re a group of community professionals who push each other to the cutting edge of our work. Every day we share our experiences, highlight new expertise, and push one another to become the best community builders we can be.

The day you join, you’ll get access to 1000+ practical tips, dozens of resources, and unlimited access to the top experts in our space. You’ll also get to see how we go about building a community amongst our own audience.

It takes just 30 seconds and doesn’t cost a thing.

CLICK HERE AND JOIN FEVERBEE

LevelPrice 
ExpertThe price for membership is $0.00 now.
Select
EnterpriseThe price for membership is $940.00 per month or $8,400.00 per year
Select
Large EnterpriseThe price for membership is $7,450.00 per month or $68,000.00 per year
Select