What Happens When The Metrics Change?
If one of your key metrics rises or falls by 20% in the next three months what will you do differently?
I’ve asked this question over a dozen times and I don’t think I’ve ever received a great answer.
Most people reply with some version of ‘we will look into it’.
This is the problem with measurement. Measurement without action is redundant. If you’re not going to do something with the data, why bother collecting the data?
If the only thing we’re doing with data is producing reports, we’re missing out on a huge opportunity to improve our communities.
You should measure your community not just to see if what you’ve done has worked, but to see what you can improve.
But this requires adopting a few simple frameworks. Here are a few we use in our work which might help you too.
Framework 1: Deciding The Right Strategic Approach
The first framework is deciding which strategy to take. This should define almost everything you do in your community.
There are four broad options as you can see below:
- Optimise – You can improve what you’re already doing.
- Adapt – You can change what you’re doing to align with key trends.
- Repair – You can find what’s not working and fix it.
- Replace – You can replace what you’re doing.
Two Questions You Need To Answer
But to use this framework you need to answer two simple questions.
- Is our community achieving its goals? Are you and your stakeholders happy with the results you’re achieving? Generally, you want to know if your community is hitting its forecasted results. The metrics you use for this might range from pure ROI metrics to engagement-level metrics (outcome metrics are ideal). If you’re not sure what your targets should be, that’s a separate challenge to solve.
- Is the environment stable? A community should always be aligned with current trends. If the environment (audience preferences, organisation needs/resources, or broader macro trends) is changing the community will need to adapt. There aren’t any fixed metrics for this – you need to be able to analyse the internal environment and audience trends (surveys help here) and come to your own conclusion.
The answers to these questions will determine what kind of strategy you should be pursuing; optimise, adapt, repair, or replace.
We’ll cover frameworks for optimising and repairing a community below.
Framework 2: How To Optimise Your Community
In the optimise approach, you are looking to improve what you’re already doing.
The two key questions to answer here are.
- What’s important to stakeholders (members/staff)? This will be surfaced in your internal interviews, member interviews, and survey results. It’s critical you know what people find valuable and important about the community. This might be different from the visibly most popular areas.
- What is and isn’t working well? What areas of the community or initiatives are generating the desired results/growing and which aren’t? This can include entire categories, superusers, or ideas.
You should begin with a deep evaluation of your current community and then decide where to invest your time to achieve the best results.
The outcomes, as you can see below, will decide where you need to invest more time and resources, what you need to tweak and change, what you need to keep doing, and what you should stop.
A key principle here if your investments must be matched against your divestments. If you want to spend more time on something, that time has to come from somewhere.
This is also where your competitive analysis can have a big impact. You can see what’s best in class (or where you’re lagging behind) and focus on improving that.
Again we use a two-by-two matrix to evaluate not just if a metric is going up or down – but if that metric is important in the first place. And if it is, what we should do if it goes up or down?
This will guide you to where you need to invest more time and resources for the best results.
Framework 3: How To Repair Your Community
If things are relatively stable, but you’re not finding the success you want you need to figure out what’s not working and why.
The common mistake here is to begin suggesting solutions before properly diagnosing the problems. You can use data to solve any community challenge.
This is where we deploy a root cause analysis.
A root cause analysis is where you create a set of binary questions to diagnose the problem and then you work backwards to develop the solution.
If an important metric has declined, for example, you then need a data question to analysis it.
There are only two possible reasons for this.
Either members are visiting less frequently, or they’re posting less frequently when they do visit (or some combination of the two).
These are data questions you can answer to decide where to focus your efforts.
Because these are two different types of problems.
Then, as you see above, you can set up more questions to zero in on exactly what isn’t working and find the root cause of the issue to fix.
You might find posts have declined because there are fewer replies to discussions due to experts drifting away from the community.
Or you might find posts have declined because fewer newcomers are registering due to a failure of CSMs to make them aware of the community.
Again, these are totally different types of problems to solve. But it’s key to make sure you know what type of problem you are trying to solve.
The template above is just an example. The key thing here is to frequently ask why.
Summary
The key principle here is to always think about what you will do with the data you collect.
Collecting data to put in reports to send to your boss obviously isn’t a bad idea, but it’s only a fraction of the value you should be getting from your data.
Data is your torchlight to guide you out of the dark uncertainty and clarify what you should be focusing on at any given time.
If you have the right frameworks, you can use these to continually and consistently improve your community efforts. You will know what to prioritise and what to stop doing. You can ensure you’re maximising your results with whatever resources you have available.
