Three times in the past year, we’ve been asked to evaluate an organisation’s community by its new leader. And I’m frequently surprised by how often our results, those rooted in data and evidence, differ so greatly from those of the folks responsible for the community.
I suspect the result comes down to how we evaluate.
Here’s a common scenario.
You’ve just taken over a community you didn’t build. Perhaps you’re the new community head, VP, or CMO – community is now a part of your role.
Maybe you’ve worked your way up through the tiers of community, or perhaps you’ve been reassigned or acquired community responsibility outside of your work.
But, quite quickly, you’re going to get a lot of opinions about the community from colleagues, members, and peers.
The platform is dated, engagement is down, the team is stretched.
This naturally leads into recommended actions. Someone wants a redesign, someone wants a new platform, someone wants to shut the whole thing down, etc…
There’s a real danger here of accepting these views as fact and taking the wrong course of action entirely.
Before you do anything, you need to determine what you’ve actually inherited here.
This has to be an evidence-based process, rather than opinions. Because you’re going to be faced with one of five major strategic choices, and you need to get it right.
You can find the full version in our Community Strategy Guide. But, for simplicity, we can strip everything down to three simple questions.
If you’re following the ‘first 90 days‘ formula, I’d recommend undertaking these three questions and then making the one critical decision about what to do next.
Key Lesson: Don’t Start With The Platform
I’ve noticed that the less familiar new community execs are with the concept of community, the more likely they are to focus on the platform.
That makes sense; it’s the most visible aspect of community.
And if you’re used to modern CMS or LMS experience, community platforms can look pretty ugly.
So the instinct is always to think about updating it. This is a problem for two reasons.
- Migrating a platform is significantly more difficult and costly than you might think.
- It’s rarely the root cause of major problems in a community.
This doesn’t mean you shouldn’t ever do it, but it shouldn’t be the first thing you look at.
More often than not, the community is set up fine, and the problems lie in how it’s run, who it’s for, and whether anyone ever agreed on what it was supposed to achieve.
You can’t know that until you’ve looked.
If you do end up needing to compare options later, our platform comparison tool is there. Just don’t lead with it.
It’s far better to focus on the real questions first.
Question One: What Is The Experience For Members Really Like?
I’m constantly amazed how frequently those closest to the community don’t objectively evaluate the experience of being a member.
Go through the community the way a member does. Sign up as a new user. Ask a question. Try to find something. Read the last week of activity. Hardly anyone does this on a community they’ve just taken over. You will quickly become familiar with the experience and likely problems.
If you want a complete picture, follow FeverBee’s benchmarking process across a set of categories: liveliness, how content is organised, the quality of answers, events, navigation, look and feel, the mobile experience, and whether there’s a clear call to action.
Each one gets a rating, so you can get a clear scorecard. Our Community Benchmarks & Standards set out what “good” looks like in each category.
A couple of specific things to think about:
- How quickly do questions get a useful reply? If your median is slower than 24 hours, members have usually gone elsewhere for the answer before it arrives.
- What share of questions ever get resolved? In a healthy support community, you expect a decent proportion to reach a marked solution. If few questions are being marked as resolved, that can indicate a problem.
- Where do new members drop off? Onboarding is where most of the loss happens, and most onboarding effort is spent in the wrong place. Our onboarding journey resources cover what actually moves the number.
It’s worth reviewing key steps to improve the community experience and some common misunderstandings about building successful community experiences.
If you don’t know whether the current community experience is good, it’s very difficult to decide what to do next.
Question Two: What Do Members Actually Want?
I can’t stress enough the importance of engaging directly with members to find out what they really want. Communities are often launched upon assumptions – which may or may not be true – but which are then never updated.
Many of the assumptions about what members want, how they want to engage, and why they want to engage are remarkably out of date.
Two weeks of listening here will save you a year of guessing.
Do three specific things here:
- Interview eight to twelve members. Not a survey, actual conversations. Include people who left, people who lurk, and a couple of your most active members. You’re listening for the goal they were trying to achieve and where the community helped or got in the way. Our member interview questions give you a script to start from.
- Run a short survey to put numbers behind what the interviews surface. Keep it tight and behavioural, not a satisfaction poll. Our example member survey is built for exactly this.
- Segment what you learn. You’ll usually find the community is serving three or four different audiences, not one, and each wants something different. Our segmentation template and persona template help you set that out so the rest of the organization can see it too.
You end this question with a clear picture of who your members are and what each group is really there to do. You’ll use it in every decision that follows.
Aside, a common mistake is to only speak to the most engaged people in your community (these are the folks most likely to respond to requests for interviews). It really helps here to speak to the desired audience who aren’t participating in your community.
Find out what their needs are, the level of their desire, frequency of need, where they go today, and why. The results illuminate what you aren’t doing today but should be considering before launching any other programme. You can map it out like this:
Reflect carefully here if the community is truly delivering an experience that matches the needs of your members today – and to what extent is that need evolving?
Question Three: What Is This Supposed To Produce?
There’s no shortage of organisations investing six- to seven-figure sums annually within their communities without ever being entirely clear on the desired outcome.
In some organisations, stakeholders we speak to highlight competing objectives for the community. In others, people give vague ideas (support/marketing) without being crystal clear in the role the community plays.
The key here is to ask the people who fund it what they think they’re buying. You’ll often get three different answers from three stakeholders, and that gap reveals a communication and alignment problem to resolve later.
You’re checking three things:
- Is there a valid north star? Does the community have a clear north star, and is everyone aligned on it? Does it clearly support deflection, retention, product feedback, advocacy, etc?
- Is the mechanism for those results clear? How specifically does the community achieve that goal? Does it change how many support tickets are resolved? Do members learn how to do more with the product? Does it improve member sentiment about the company or its products, etc?
- Is there a valid measurement process? Is there a measurement model in place that would survive the sniff test? How is attribution calculated? Is the data accurate and good? Are the results clearly displayed for everyone to see?
You don’t need the full measurement model in ninety days. You need to know whether the community has a model and whether the model is reliable.
Our 46 questions for uncovering community goals, our guide to building a community dashboard, and our community ROI guide take you further when you’re ready.
The dream community dataset shows what a complete picture looks like.
Create Your Problem Statements
Based on this, you should be able to turn your observations into some clear problem statements.
There are some good and bad versions of this.
Bad is: “engagement is low and the platform feels dated,”
That’s a vague opinion. Good names the segment, behaviour, evidence, and cost. For example compare:
Bad: “Onboarding needs work.”
Good: “Around 60% of new members never return after their first visit. They arrive with a question, don’t get an answer within a day, and find it elsewhere. That’s roughly [X] members a month we acquire and immediately lose.”
The latter is a measurably clear problem to act on.
Likewise,
Bad: “Members aren’t very engaged.”
Good: “Around 30% of members told us they want to connect with peers in the same role, but there’s no way to find people like them, so they post once and leave. Our most valuable segment has no reason to stay.”
Again, it’s specific, measurable, and clear that there’s a problem which needs to be resolved.
One more example:
Bad: “We’re not showing value to the business.”
Good: “The community resolves product questions that would otherwise become support tickets, but we’ve never counted them, so at renewal we can’t say what it saves. Support leadership currently assumes the answer is zero.”
Your numbers might be your own, but the shape is what matters. You get the idea – give clear, specific problems that affect an identifiable audience.
Don’t worry about solving them yet; properly identifying the problems is more important at this point – and most problems naturally point to obvious solutions.
What Type Of Strategy Do You Pursue?
Once you have all this information, you can decide what kind of strategy makes the most sense for you.
You can learn more about which type of strategy each is here.
Essentially:
- Optimise what you already have. If members’ needs aren’t changing and the environment is stable, work on incremental improvement based on your results above.
- Repair. If the community isn’t successful, but the environment is stable, then identify the issues and repair them.
- AI-Ready. If the community is doing well, but the environment is changing (noticeable due to AI), align the community to those goals.
- Community Belonging. If the community isn’t doing well and the environment is changing, one of the two approaches is to focus on belonging. In a world of AI, belonging can be scarcer.
- Community Everywhere. If belonging isn’t the right answer, the final approach is to engage members where they participate. This is a more decentralised approach.
If it helps, use your research above to complete this short survey.
This will guide you to the strategy options which may make the most sense to you.
After this process, you should be able to walk into any meeting with a clear direction and vision for the community – and defend it with real data and coherent thinking.
Or Hand The Project To Independent Consultants
While you can run a short version of the process above yourself, you will come up against two problems.
- Limited time. Interviewing enough members to be sure, running the survey, and benchmarking your community against others in the same category is a lot of work, and it lands on top of everything else in your first quarter. It’s the part that often gets skipped. You likely need someone you can outsource some or all of this project to.
- Bias. You own this community now, so you have a stake in the answer before you start. Without meaning to, you’ll lean towards the evidence that fits the plan you already like, and towards protecting the parts you’ve inherited. Colleagues and members often sense this and align their responses to what they feel the interviewer wants to hear.
This is why I believe it typically helps to hand the diagnosis to us and get a clean, unbiased evaluation and the problem statements back. It saves time and typically delivers better results.
If you’ve inherited a community you know is underperforming but can’t yet say why, that’s where we start.