Back in March, I shared some data on Twitter where we definitively proved the ROI of an online community by removing it from search channels.
1 – I want to share some data from a client we worked with last year where we did something that I think was a little clever to definitively prove the incredible value of their community.
But first, some background….#CMGR
— Richard Millington (@RichMillington) March 1, 2021
To the best of my knowledge, no one has ever done anything like this before and it was the persuasive data we’ve ever collected.
It’s one thing to develop a formula in theory, it’s another thing entirely to deliberately cause traffic to plunge and see what happens.
Now I’m happy to say we’ve published the data in longer form in this article on Harvard Business Review.
If you want to learn exactly how we undertook the experiment and the full range of results we discovered (they are amazing), read the article at HBR.
There was a major fire in my building recently (I’m fine).
The tenants, angry with the property company which manages the building, quickly switched from the company’s official hosted community to a private WhatsApp group. We needed to discuss fire safety issues, legal problems, and next steps.
Likewise, I recently decided not to work with a debt management company that wanted to build a community for people to share debt-saving tips. A little primitive research suggested almost nobody wants to participate in a debt-advice community hosted by the very company they owe money to.
Sometimes you simply can’t build a hosted community because of the nature of who you are and the relationship you have with the people you want in the community. If you have power over them and/or they are angry with you, they want to have their own place.
No one wants to join a dieting community hosted by McDonalds. Trade unionists aren’t going to participate in a community hosted by industry. And we’ve seen many examples in major tech companies recently where employees created private groups to discuss major concerns.
I’ve met many, many, organisations over the past decade who ignored this and still tried to host their own community. They always wound up disappointed.
But just because you can’t host a community doesn’t mean you can’t proactively support and engage with a community. You can help encourage the community (or communities) which do exist, provide support where needed, and most importantly, take the time to listen and address their concerns.
It’s not as glamorous as hosting a community perhaps, but it’s far more useful for your audience.
I’m not sure ‘like’, ‘upvote’, or ‘kudos’ buttons on answers are especially helpful.
Compare two questions:
Question 1 – ‘My iPhone isn’t charging, can anyone help?’
Question 2 – ‘How do I Import FLAC audio files on Adobe Premier Rush on my iPhone?’
Let’s imagine question 1 gets 2,000 visits and 10% click ‘like’ on the answer, whereas question 2 gets only 100 visits but 90% click ‘like’ on the answer.
The answer to question 2 is clearly infinitely superior to the answer to question 1. But the response to question 1 will still get more likes simply by the volume of traffic.
Counting ‘likes’ doesn’t reflect the success of the community or a question, it often better reflects the volume of traffic which visits the community or questions.
You need a denominator. What % of people found a response helpful? You could calculate this by measuring the number of likes by the number of people who visit the answer. But the majority of people don’t bother to click ‘like’ at all and higher-volume questions tend to attract less-targeted traffic.
A better way is simply a ‘yes/no button’ (i.e. did this answer help? yes/no). You can find this on many knowledge base articles today. This gives you a great benchmark to work with and improve over time. It also lets you identify popular questions with poor answers you can improve.
Early on in my consulting career, I advised a client to use a taxonomy and categorisation system for the community which was distinct from their other content articles and internal structures.
In the short term, this was better for the community. But seven years later, this made it difficult to do really simple things.
For example, if you want related discussions to show up alongside relevant content articles, how will your system know what’s relevant? How will it know what keywords to match? Because I had advised a different taxonomy, there was no metadata to connect the two.
If you want to automatically route relevant articles, know what products members are talking about, or do anything which involves connecting two systems, you have to think through precisely how that will work and plan in advance for it.
Generally speaking, it’s unlikely a company is going to change its entire taxonomy and data structure to suit your community. So if you want the community to integrate with other systems, you need to adapt the community as best as you can to your current systems.
A common idea is to launch a community, get a few members, and wait for members to spread the word.
That can happen, but it’s rare. It’s rarer still in private communities which aren’t open to the public.
Be honest, how many communities have you recommended to your peers in the last month? (or even in the last year?). It’s probably not many (if any!).
This is why Rand’s post on amplification content is worth a read. Many of the motivations/reasons for amplifying a content Rand lists are the same I’d list for promoting a community. To adapt Rand’s reasons slightly, these include:
- Novelty. It’s a completely brand new and surprising idea for a community (a typical forum for people to ask and answer questions doesn’t cut it).
- Belief reinforcement (i.e. it validated your belief). The community concept or content posted within the community strongly supports a closely held belief and connects people who strongly support that belief.
- Relationship with the founder. Members know the founder and want to help the founder.
- Controversy. Disputes between members that resonate on a broader scale (or even a good debate).
- Familiarity. It hosts many of the well-known names in the industry.
- Rankings. The community offers a codified ranking of the best successes within the sector or top people within the community.
- Ego. Members have their egos invested in the success of the community. Either the community is about them, they’re the founding members etc…
This isn’t a comprehensive list, but it’s a good place to start.
While you might attract organic growth via search over time, if you want members to talk about it you need to give them a reason too besides the community simply being ‘really good’.
In a current client project, we’ve been launching a new, private, community in a niche field.
Given the nature of the topic, we needed to invite and approve everyone who joined.
We quickly noticed a problem. Regardless of how quickly we approved new accounts (most were approved within an hour), many approved members weren’t returning back to visit the community. That short delay was having a big negative impact.
We tried @mentioning members and nudging them from within the community, but it wasn’t having much of a difference.
So we tried Discourse’s improved invite system instead. Essentially, anyone who joins via a specially created link is pre-approved to join the community. We began using this link in every invite instead of the generic community link.
It’s early days, but so far the results have been close to a 100% success rate. Removing the wait to be approved has a big impact upon someone’s likelihood of visiting and participating in the community.
If you’re building a private community and your platform lets you create a pre-approved list (or pre-approved emails/domains etc..), I’d suggest you use it.
Every couple of months we scrape and analyse data of various communities to see how they’re doing.
The method isn’t perfect. Scraping doesn’t capture private discussions and some organisations tend to remove posts over time which influence the results. But it does give you a vivid picture of how a sample of top brand communities are fairing.
Posts Per Month Per Community
Looking at the number of posts per month per community, it’s clear COVID did cause a spike of around 50% in most communities. However, as time progresses, the number seems to be returning to the baseline. (p.s. If you’re running a brand community, 4k – 5k posts per month seems to be a good target).
Breaking this down by community, we can see wild fluctuations depending upon the community sector.
We can see wild fluctuations in the data based upon the community. However, other than a slight uplift, there aren’t any major trends. Although the increase in EA Games is worth noting.
No. Active Members Per Month
Next we look at how many members are engaging in the community. This is the number of unique usernames who made 1 post in the community within the past month.
Overall, there seems to be a slight upload trend before the pandemic hit. The pandemic caused a spike as a flood of new members began asking questions in the community instead of other channels. This spike has largely been sustained with a slight downtick in February (28 days).
Note the median of around 1500 active users per month as a reasonable target for most enterprise brand communities to aim for.
We can break down this by each community in the sample too.
This chart shows the importance of focusing on the median rather than the mean. Some communities had a huge COVID surge (HP had more people setting up home printers/laptops), EA Games presumably had more people playing games at home.
Overall we’re seeing the pandemic has driven more people to use communities and a small percentage of those seems to be sticking.
This hasn’t massively increased the volume of posts members are making when they do visit (i.e. the communities themselves aren’t becoming more engaging), but the pandemic has accelerated the relatively small trend towards members participating in communities.
I love this from Apple.
This is exactly the way things should work.
- First, you browse articles for an answer (you read the manual).
- Second, you ask the community for help (even better, you can click an option to see all the existing questions about the topic).
- Third, you contact support if nothing else solves your problem (support agents are tackling the tougher questions).
What’s so remarkable (and so frustrating) is how few brands do this today. Why is there not an option to ask the community for help on every single article on the site? It’s such a huge win at such little cost.
You can find the video here (edited out some technical issues at the beginning).
You can watch the video and make up your own mind on the value of our arguments.
It’s worth noting the popularity of the format. I’ve hosted hundreds of webinars over the decade and never had as many registrations as this one (this also speaks to the popularity of David). Nor has any previous panel or webinar had the quantity and quality of engagement of this session.
I think there’s a few things going on here:
1) Testing a new format. We certainly didn’t invent this debate format, but they don’t seem to be widely used. Yet every community has contentious issues with competing sides. I suspect there’s a deep level of fatigue with typical podcast interviews. Interviewees aren’t typically forced to vigorously defend their claims.
A debate (where the audience selects a winner) flips that on its head. You know you’re going to be pushed hard so you need to justify your arguments better. This increases the quality of the discussion (as long as you can keep it civil). There is no reason you can’t select two known figures in your community to debate competing issues (it might even resolve the topic from spilling over into other areas).
2) Tribal nature. Debates with a winner tap into a tribal nature. People get to pick a side and debate the claims of one another. The quality of engagement in the chat was extremely high (and frequently funny) throughout. It never felt like just me and David debating, it felt like everyone else debating too.
3) Real-time feedback. The value of having the chat window allowed every attendee to be a real-time fact-checker. This also makes the audience part of the debate and surfaces higher-quality information than we would get without the chat. This provides the entire community with a great role to play in the debate.
I’m still debating (aha!) whether to do more of these. But I definitely think more communities should explore it. The cost is minimal, the preparation isn’t especially high, and you might find you have an engaging format your members love.
A short-sighted approach to building a community is to start with any activity you can get and then try and improve it.
This almost never works. The people who would make high-quality contributions will be turned off by low-quality activity. No one is going to add a thoughtful analytical piece to a site filled with memes and off-topic discussion. Soon, people will notice they’re not getting much value and move on.
The better approach is to demand high-quality contributions right from the beginning. Be clear about the expectations. Reject and remove contributions (even contributors) who can’t meet that high standard.
It’s much harder to get started, but once things get going quality begets quality. The key is to create and sell a compelling vision. A vision that highlights the benefits of this community, the unique role members can play within it, and the urgency of doing it now.
Low-quality engagement is a trap. It might seem like metrics are improving but you’re just digging deeper into the quicksand. The real magic in starting a community is in creating and executing a powerful story that attracts and resonates with the very people you want in your community.
In theory, you and your team would follow clear guidelines to make decisions about whether or not to remove members from a community.
Yet a study of Wikipedia admins revealed admins/moderators with ‘low trust’ in strangers were 81% more likely to enforce the community guidelines.
Very few moderators follow strict guidelines. This is a good thing. Community guidelines only specify the offense, they rarely consider the intent.
Intent and context matters. You might ban swear words, but would you enforce the same punishment upon someone who said “fuck you” and someone else who said “that was a fucking amazing answer, thank you so much!”
But considering intent is also where subjective bias (i.e. our past experiences) shapes our thinking. If your experience is ‘people are generally bad, new members can’t be trusted, and everyone is a potential troll’, that’s going to be reflected in more frequent enforcement of the rules.
I know one community manager who moved from a gaming community (where problems were more frequent) to a professional community of scientists and did tremendous harm in enforcing arcane rules upon newcomers.
Two things are important here. First, be aware of the benefits and the costs of your past experience. Every community is unique and your past frame of reference in how members will behave might be very different from how your new members behave.
Second, when hiring a community manager, get a sense of how they think of members and problems in the community. Ask if they typically trust newcomers in a community and what percentage of members caused issues in their past community. Recruit people with the right frame of reference for making decisions in your community.
You’re in a bubble.
When you go to work (or log in to work), you’re interacting with people who are highly engaged with your brand.
When you talk to members, they feel highly engaged with what you’re saying.
When you see your organization featured in the news, it reinforces how important your organisation is within your field.
If you solicit the thoughts and ideas of prospective members about the community, they will probably tell you it’s a great idea. They’re picking up on your cues and telling you what you want to hear (and there’s no downside).
Almost everything you see, hear, or do within your work is likely to further confirm that people are far more engaged with your brand than they actually are.
This often leads to two common problems.
The first is believing a community will be easier to get started than it actually is. Instead of spending weeks, even months, developing and refining a magnetic community concept, we instead spend our time on the technology and assume people will naturally join.
The second is believing the community should have more engagement than necessary. Why aren’t people who clearly love your brand more engaged in your community? What’s wrong?
You need to pop this bubble quicker.
Think about it this way. How many communities do you participate in? 2 to 3? Maybe?
Particularly, how many brand communities do you participate in? I’m going to guess it’s not many (if any!). There’s a good chance you don’t participate in any communities for the topics (or brands) you’re most passionate about. And if you don’t participate in these communities, why would you assume your audience does?)
The reality is there is a ferocious battle for the time and attention of your audience. Your community is coming up against everything else your members can be doing at that moment. If members don’t seem to care that you launched a community, it’s probably because you’re not particularly interested in the community launch of a brand you like.
This is a long way of saying that it’s far harder and takes a much bigger emotional pull to persuade people to stop watching Netflix and spend that time in your community instead.