Three Very Different Ways To Analyse An Online Community
Most people work from a simple assumption (e.g. “higher levels of activity per member, an increase in retention rates”). This means you measure activity per member and design your engagement activity to maximise activity per member.
The downside is you haven’t proved if the relationship is true (i.e. does it correlate well?), nor the influence levels of activity have on an individual’s retention rate (does it account for 100% or 5% of retention rate increase?), nor whether the relationship is linear (does it drop off after a certain level?).
You could blindly be pursuing more activity when data might show you that 3 posts per member, per month, is enough.
A better approach is to test a falsifiable hypothesis (e.g. sample members by levels of activity and compare this with customer retention rates to prove if the relationship exists and how influential the level of activity is in that relationship). You could then focus on increasing the level of activity from a specific segment to see if retention rate among that segment rises (this isn’t a natural experiment, but it’s still good). You might find that there is a relationship between increased levels of activity and retention, it’s nonlinear. After 5 posts per month, there is little impact.
Now instead of trying to get every member super active, you focus on ensuring they make 5 good contributions per month. This changes how you work a little.
An even better approach is to run a regression analysis to identify which variables correlate with increased levels of member retention. You might find that increased activity accounts for a 27% increase in higher retention rates. This also highlights other key variables. You might find direct messages between members have a 25% influence, opening newsletters have an 18% influence, and adding a profile picture have 10%. You can now test these relationships and build a mechanistic model.
Now instead of trying to maximise member activity at all costs, you might spend more time on newsletter subject lines and content, persuading members to add their profile picture, and ensuring members befriend each other to increase the number of direct messages.
This isn’t easy to do, but that’s exactly what makes it valuable. You can build a specific, tested, model that will show you exactly what you need to spend your time on to achieve. This lies beyond the endless hunt for more engagement. It’s where you can take your work to a more advanced, strategic, level.
Make your own comment on this post at FeverBee Experts
Hi Rich,
I find that your post is also in line with the benchmark results from activity and social cohesion made the team of Laurence Lock Lee. His results show that it can be actually dangerous to only focus on activity if what you are trying to get is valuable collaboration.
"Using data from more than 135,000 people across more than 20 organizations, we have uncovered the following relevant key insights:
1. There is no association between activity levels and response rate. We expected that the more activity a network has, the more likely it is that posts are replied to. Getting a response to a post is obviously essential for collaboration — otherwise you are talking to yourself. So it was a big surprise for us to discover that there was no association between activity levels and a response.
2. Social Cohesion (measured as the intensity of reciprocated interactions) is the single measure that most differentiates the organizations in our benchmarking sample. Measuring social cohesion is equivalent to measuring your blood pressure — it is an essential health indicator. There is no association between activity levels and social cohesion. We found examples of highly active networks with poor social cohesion. That’s like having lots of people at a party, but the dance-floor is empty.
3. After the initial launch hype, the average adoption rate is around 24 percent with the highest being 63 percent. C-Suite executives are looking for a fully inclusive collaboration solution. Adoption rates are still a major challenge."
The lessons are clear, we should create our engagement strategies based on what we want to achieve, not just generating more activity on the community or network.
Any idea how to measure social cohesion?
I’ve got some hints from that article, but I think that we have two options:
Here are the hints I talked about written in his interview by Virpi Oinonen:
“Our ‘Social Cohesion’ measure is a simple proxy for ‘reciprocity’. It simply measures the degree to which your interactions are reciprocated by others in your network e.g. you reply to my posts and I reply to yours. Aggregating individual results to the whole enterprise means that we have the ability to assess social cohesion at all levels of an organisation. In network science, reciprocity is a fundamental and key indicator of trust and therefore performance.”
I’m not sure social cohesion (by that metric) is the best metric here.
Let’s imagine I ask a question in an internal community and someone with
the right expertise answers the question.
By the social cohesion measure, I should then answer any question they then
ask.
The problem is I might not know the answer. In fact, statistically, the
odds of me having the specific answer to a question from someone that
answered my question must be extraordinarily low. It feels like this is
measuring the wrong thing. It doesn’t matter if the same person answers my
question, only that my question receives a good, quick, answer.
I think better measures here aren’t social cohesion by that definition, but
I’d suspect more along the lines of speed, quantity, and quality of
answers. An internal community has to be seen as more valuable than any
other communication method.
Thanks for the replies. For the Q&A section, we are focusing indeed in quite specific metrics:
Questions asked, solved.
Existing questions (and their answers) appear as you type.
Concerning efficiency:
Concerning connections:
The social Q&A tool we are trying is called Starmind and it brings the statistics in its service, the goals we fix are the ones that they obtain with other networks with over 5K users.
I know that Philips gets 50% of new questions answered within 1h, but they have 200K employees.
I would think that the social cohesion could be more interesting with regards to blog posts, micro-blogging, or comments on documents.