We believe organisations should be making data-driven decisions about their communities.
This means you need to begin with really great data.
I want to share five techniques we use to evaluate a community.
1. Community UX Testing
If you want to evaluate the community experience, you can use your own judgement. But the way you look at a community isn’t the way members engage a community. It’s far better to undertake really good UX testing instead.
This means going through the community with members and seeing the community through their eyes. Specifically, this includes:
- Get members to share their screens.
- Give them tasks to do (ask/answer questions. search for info).
- Evaluate where they get stuck or confused.
You should be able to quickly identify a range of issues and take steps to solve them. Prioritise the biggest problems and develop solutions.
| Issue | Solution | Severity | Effort | Priority |
|---|---|---|---|---|
| Doesn't realise the login option is also the registration option. | Easy to tweak to login/register. | Severe | Low | High |
| The 'forgot password' feature isn't working. | Needs a developer to review the process and resolve it. | Severe | Low | High |
| Community is 'overwhelming'. Members don't know where to ask questions. | Revamp homepage with separate instances for members depending upon profile data | Severe | Moderate | Medium |
| Members can't find the posts they've recently made. | Need to show members the recent posts they've made on the homepage when they visit and in the member profile | Medium | High | Medium |
| Members can't find the latest information about products. | Need a separate signposted area on the homepage and on discussion pages | Medium | Moderate | Medium |
| The registration process takes too much time to progress through. | Need to reduce redundant pages and information we ask from newcomers. | Medium | High | Low |
| Members are unable to connect with people like themselves. | Need to create a member directory and solicit metadata from members | Low | High | Low |
You should be able to build a list of solutions like this quickly.
Resource: How To Undertake Community UX Research
2. Analysis of Community Management Performance
It’s a good idea to benchmark how the community is managed. This typically means looking at a couple of things.
1. Review the community against competitors by avg. time to first response, response rate, and accepted solution rate. You can also do this by category for deeper insights.
2. Benchmark the quality of responses against training/standards. Your community team should be trained to respond at this level. Pull a sample of responses and see how they compare against these standards.
3. Post questions and evaluate responses. We sometimes post questions in the community and see what the responses are like.
4. Test spam/troll processes. You can review how quickly spam is removed and how quickly trolls are addressed.
Resource: How To Improve Your Engagement Skills.
3. Acquisition and Onboarding
You should go through and experience the entire member journey yourself from scratch. Remember that in many communities, members don’t arrive at the home page. The first interaction might be a good search. So begin there.
Our evaluation of onboarding would typically include:
- Join the community ourselves and go through the process with prospective members. You can use the same approach as UX testing above. See where people get stuck or are confused. Consider how long it takes to complete.
- Look at conversion data and where the drop-off points are. Look at the entire onboarding journey and see where the majority of members are dropping out of the process. Use this as your baseline and suggest improvements.
- Interview newcomers about their experience. See how they felt about the post-registration experience. Do they even recall any of the emails or messages they received? Were they useful? Could some things be removed?
Tip: Most of the time you can remove the majority of steps and increase the retention rate significantly.
Resource: Case Study: How FeverBee Tripled A Community’s Retention Rates (scripts, templates, and examples)
4. Top Member Program
It’s very hard not to be biased about your own top member program (aside, this is often why getting independent consultants like ourselves can help). Some of the things we would look at include:
- What % of questions are answered by top members? This is a good indication of how effective the program is.
- Who are the best performing top members? Use this bubble chart approach as before to see which members merit the most attention and which can be removed. Compare top members by # answers, speed of response, and accepted solutions.
By removing a few top members from one program we increased performance significantly.
- Interview current members to identify needs/desires /satisfaction. This will let you build a detailed top member persona to cater towards (hint: face-time with staff is usually high on the list).
- Review recruitment and documentation. Review the documentation of the community. Is it clear how to join, what members get, and how the program operates?
You should typically be able to identify concerns and improvements which can be made here.
Resource: Community Superuser Programs
5. Rewards / Benefits
We analyse whether the community is offering the right rewards to members and whether members feel they are getting these rewards. There are plenty of ways to review this. Some of the easiest include:
1. Use surveys where members can identify and prioritise what they want from the community. Make sure members prioritise what they want. That’s the key.
Ask members how important or unimportant factors like the above are.
2. Review positioning vs. other channels. Does the community occupy unique positioning in the eyes of members vs. other channels? Is this aligned with what members need?
Every community should occupy a unique value proposition for members.
3. Use interviews to identify specific requirements. Use member interviews to compare the community against pre-existing personas and ensure each is getting the value they need.
This should help you identify if the community offers the right rewards to attract and retain members. It should also help you determine if the community is delivering on the promised rewards.
Resource: Common Member Survey Questions
Resource: Common Member Interview Questions
Summary
You can use this data to create a simple evaluation showing how the community is doing today and where the focus might want to be.
A simple summary of a detailed community evaluation.
The reason why so many strategies fail to achieve their goal is they aren’t built upon high-quality data. If you have the right data, it becomes a lot easier to identify the right solutions and prioritise what you should be working on.
Get Consultancy Support
If you don’t have the time or expertise to do it yourself (or need an objective opinion), get consultancy support.
We have many clients who want updated benchmarks each year to evaluate their performance and track progress. We would be excited to hear from you too.
Resources:
