The Perfect Community Metric – Finding Your North Star

A north star metric is a single metric that represents the success of your community.

Ideally, it’s a single metric all stakeholders can track to see if the community is delivering more value than in the past. If it goes up, you’re doing well. If it goes down, things aren’t going so well.

If you find the right metric, it’s a lot easier to get more support and build more awareness of the community. However, finding the right metric isn’t easy. Every metric has limitations and it’s not easy to find a metric which reflects your organisation’s unique goals and constraints.

In this post, I want to share some of our thinking about finding your north star and finding the perfect metric for your efforts.

 

Why Finding The North Star Is Painfully Difficult

Before we begin, it’s important to be mindful of why this process is painfully difficult. Most previous efforts to find a north star metric suffer from four problems.

1) They don’t reflect value. Many metrics show increases or decreases in activity, but not whether the activity is important or not. This leads to the ‘engagement trap’ problem.

2) Gaming the metrics. Many metrics are easy to game. This undermines the entire value of the metric. Good metrics always need to be accompanied by good judgment (and ethical behaviour).

3) They are driven by external factors. Many metrics change not as a result of community, but as a result of external factors outside of the community team’s control. You shouldn’t be held accountable for metrics you can’t directly influence.

4) They aren’t comparable to other metrics. It’s hard to know if the metric matters if it can’t be compared to any existing metrics. A community-focused metric is great, but how do you compare that to a return on marketing spend?

Be mindful that the challenge isn’t to find a flawless metric. There isn’t one. The challenge is to find out which flaws are acceptable and communicate these well to others.

We will go deeper into each of these flaws and make suggestions about which kinds of metrics might make sense.

Resource: The Tyranny of Metrics
Resource: The Engagement Trap
Resource: Don’t Use Bogus Metrics

 

The Journey Is As Important As The Destination

The process of finding the right north star metric is as important as the metric itself.

You shouldn’t come up with a metric by yourself. That’s a surefire way to set yourself up for endless years of arguing why the metric matters. Instead, you should treat this as a journey of discovery and close collaboration where you listen to the thoughts of others and bring people along on the journey.

Everyone should feel they were involved in finding the final metric. If you don’t do this, you might face a concerted effort of people to unpick the value later.

You can see a typical process to find the north star metric below:

ROI Flowchart Community

At every stage of the process, you need to build greater understanding and keep people informed. This might begin with knowing if a precise dollar value is needed. But that’s far from the end of the journey.

Be mindful that this chart isn’t prescriptive – it’s not designed to cover every possible type of community (non-profits, partners, developers etc…), but it might be a good place to begin investigating the opportunities.

 

PART ONE – FINDING THE DOLLAR VALUE

Is A Precise Dollar Value Required?

The first challenge is to ascertain expectations. Perhaps the most important determination is whether you’re looking for a dollar value return or not.

The method to find a precise dollar value is completely different than if one that has a different goal.

If a dollar value is required, you need to develop and explain a methodology that leads to a precise calculation. The end result is a dollar value return which will likely be compared to other departments to determine if the community merits more investment.

Also, if a dollar value is required, you will need data not just from the community but from whatever systems store the spending and behaviour of customers. You need to combine community data with your CRM data.

If a dollar value is not required, you have more flexibility to design an approach that reflects the unique value of your community. If a dollar value isn’t required, it’s possible to determine value just with community data.

In this approach, you can find a metric that perfectly encapsulates the impact of the community – but might not assign a precise dollar value to the community. This is like the difference between measuring the value of a branding effort vs. a dollar ad spend. The former requires metrics that either show revenue driven by ads or a change in feelings and perceptions (usually acquired by questionnaires).

You begin the process then by answering some critical questions.

These questions include:

  • Who will be looking at the metric?
  • What do they want to know about your community?
  • How will they be using the metric?
  • What will good or bad look like?

Once you’re armed with answers to these questions, you can begin to figure out what kind of metric might make the most sense. Crucially, are people looking for a precise dollar value or a broader sense of impact?

For now, let’s assume a dollar value is required.

Understand The Purpose

The next step is to determine the purpose(s) of the community. We’re not talking about specific goals here, but a broad understanding of why the community exists. This can be singular or there might be multiple reasons. Often the answer is suggested by whichever department the community reports to (which is why shifting departments is such a big deal, the purpose of your community will also change).

Some simple questions to ask here might include:

  • Which department does the community report to (and why?)
  • Who was behind the community’s launch and what was expected?
  • What are the expectations stakeholders have of the community?

If no one seems to know (or seems reluctant to own the decision), it’s often a good idea to host a stakeholder workshop to gain some sense of alignment. This is where you guide members through the process of prioritising what is the best match for a community.

You can see a typical outcome of the process below.

Community Gaining Alignment

You don’t need specific metrics at this stage, but you should have a fairly clear purpose (or set of purposes) to define how the community helps. Is it support? Attract new customers? Retention? Or some other benefit?

Resource: 46 questions for uncovering community goals
Resource: A Framework For Hosting A Successful Workshop To Gain Alignment On Community Goals

The Four Methods Of Finding A Dollar Value

Once we know the purpose, we can start finding the right methodology to calculate community value to a precise dollar value.

You can see the four options below:

4 methods to calculate community value

Each method has some obvious pros and cons. You have to decide between precision or simplicity, correlation or causation.

The former is a question of capabilities, the latter is a question of preferences.

Let’s go a little deeper into each method.

Controlled Experiments

The gold standard is to run an experiment.

However, this rarely happens because experiments usually require:

  • An analyst with experience in running experiments.
  • 3 to 6 months to plan and execute.
  • Removal of access (or not providing access) to a significant percentage of your audience.

If you don’t have the capabilities, resources, and support for this, it won’t be possible to run an experiment. This means you can only determine correlation, not causation. Also, be mindful that an experiment gives you data from a single point in time. If things change, this data might quickly become outdated.

Controlled experiments let you make statements like:

“The community causes members to spend/do [x] more than they otherwise would”

Article: The Real Value Of Your Brand Community

Group Comparison

The second approach is to compare two groups (or more) by a specific condition.

The most common example is to compare spending or retention of members vs. non-members. Statistically, this isn’t too difficult to do. The catch is this provides you with correlation, not causation. This simply indicates there is a relationship between membership and the desired outcome, it doesn’t show which way the relationship goes (i.e. does retention drive engagement or engagement drive retention?).

Nor does it reveal whether there is a confounding variable that influences both variables as you can see below.

confounding varible diagram

This doesn’t make the data redundant (and there are more complex calculations you can do to try and ascertain the unique impact of the community), but overall it’s a useful, interesting, signal.

This lets you make statements like:

“Community members spend/do [x] more than non-members (or comparative group)”

Note: You can (and should) be precise about how you’re comparing groups. For example, you can find groups of people who closely resemble one another before they join a community and then track their spending a year later etc…

Article: What Is A Login, Post, Or Download Worth? An Analysis By FeverBee

 

Surveys and Polls

Surveys are a handy tool which can be used to determine the value of a community. For example, you can ask members about their spending before and after they join the community.

The upsides of using surveys and polls are obvious; they’re easy to set up and analyse the results.

However, surveys suffer from three common problems.

1) Getting enough responses. You usually need around 300 survey responses to have meaningful results. If we assume a 5% survey response rate, you must contact at least 6000 people.

2) Sampling bias. Surveys suffer from a sampling bias. The most active members are also those most likely to respond to survey requests. This means surveys tend to skew towards the happiest and highest-spending members. You need to use a quota system by activity level to gain an accurate picture.

3) Responses are often inaccurate. You can ask members questions like how much they purchase before and after the community (or even track purchases over time), but memories are often fuzzy and the answers might not be accurate (people might also give you the answers they think you want to hear).

This doesn’t mean surveys aren’t useful. For tracking trends over time, they are especially useful. They can also give you a precise value you can easily justify. For example, you can say something like:

“Members tell us they’re spending $343 per year because of the community”

But if you’re going to use them to calculate value just be mindful of the downsides as much as the upsides.

 

Assigning Values To Behaviors

The final approach is to assign values to member behaviours. There are two common ways to achieve this, regression analysis and alternative costs.

Regression Analysis To Assign Values To Behaviors

Let’s tackle regression (or multiple regression) analyses. This is where you look at how a change in one variable relates to another. So you might look at how changes in independent variables like asking questions, reading articles, and receiving answers affect a dependent variable such as additional sales or revenue.

This study will give you coefficients which let you assign a value to this behaviour. This helps you make statements like “We know members who ask a question spend $4 more”.

However, be mindful of the phrasing above. We’re not claiming asking a question causes the increase in spending. We’re simply asserting members who do this one thing also do this other thing. This is correlation, not causation.

This methodology offers a precise value that helps you estimate the potential value of the community and the key behaviours you might want to encourage more of.

You can see an example of this in our work with The Pragmatic Institute.

How Much Is Community Worth

Alternative Costs

Perhaps a more familiar approach is to look at the comparative cost (i.e. what would it cost to achieve the same result if the community didn’t exist?).

The most common form of this is call deflection. This is how much it would cost to support the same number of customers if the community didn’t exist.

In the standard approach, you calculate the value of a view of an answered question and multiply it by the number of answered questions. This doesn’t mean these cost savings were realised (not many people actually reduce the size of their support team). But it gives a theoretical value to a community.

Other approaches include calculating the cost of acquiring customers, and traffic, or creating the same amount of content if the community didn’t exist. First, you find the baseline cost of the work today. Second, you determine the cost of a single incident of the behaviour happening in the community. Then you multiply this figure by the number of behaviours in the community.

This will you let assert an answer like ‘the community is potentially saving $7.04m per year in deflected tickets’

 

Which Approach Should You Use?

If each approach has significant advantages and disadvantages, how do you determine which is the best approach for you?

This depends upon your organisation. You shouldn’t develop the methodology alone because it might later be picked apart due to inherent flaws. If we know every approach has flaws, it’s good to determine which flaws are already acceptable to your organisation (or department) and select that approach.

This means it’s useful to investigate which methods are used to calculate the value of other activities within that function.

This might include questions such as

  • How are dollar returns calculated for activities within that department today?
  • Do they want to adopt a similar method for calculating community activity?
  • What kind of formula or methodology are they expecting?
  • How will the results be used in the decision-making process?

Find out which methodologies are already in use, and you can adopt a similar methodology to the community.

Once you know the methodology, you can zero in on the specific value metric.

 

Zeroing In On The Metric

It’s not possible to cover every possible metric, but it might help to cover some of the most common approaches below based on the methodology.

PurposeApproachCommon Method(s)
Customer SupportValue AssignmentCall Deflection Method
(No. visitors to accepted solutions * % who state they received the answer * % who would otherwise have contacted support (* % in warranty) * avg. cost per ticket.

Monthly Contact Rate
No. members * (avg. contact rate of non-members - avg. contact rate of members) * avg. cost per contact.
Controlled ExperimentExpose or withhold community from a group

Remove community access from one or more product groups and measure the increase or decrease on customer support tickets.
Customer RetentionPaired GroupsMember vs. non-member retention rate
Avg. annual retention rate of members - avg. the annual retention rate of non-members * no. members * avg. value of a customer.

Trial completion rate
% trial completion of new community members - % trial completion of non-members * avg. value of completing trial (CLV) * no. new community members.
Controlled ExperimentInvite one group, block another
Invite one group of prospects to join the community and hide the community from another. Compare the results after a fixed time frame.
Customer AcquisitionValue AssignmentTrack signups from the community
Calculating the avg. lifetime value of members who joined the community prior to becoming a customer.

Track conversions from the corporate site
Conversion rate of visitors to corporate site * visitors from the community.

Track direct sales
Direct sales of products and upgrades promoted to the community.

Track value of leads
Number of leads sourced from community sources * avg. value of a lead.

The right metric for your community might not be above. The key, however, is to engage with others to calculate the right metric for your organisation.

If you need help here, drop us a line.

 

PART TWO – IMPACT

If the value refers to a dollar return, impact refers to everything which isn’t a dollar return.
Much of the work of the community sits several levels beneath calculable value.

For example, if a salesperson uses the community to gather testimonials, how would you quantify the value of that? You can’t attribute the sale to the community, yet few organisations want to do a test comparing presentations with testimonials as opposed to those without. Yet, you know it has an impact.

If you’re not under pressure to show a dollar value return, you have a lot more flexibility to develop a method that truly reflects the impact of your community.

But be mindful of a potential problem here. The further you drift from a dollar value, the more likely it is people will question your metric. Everyone understands dollars, fewer people understand impact.

The challenge is to strike the balance between metrics that perfectly encapsulate the value of the community and those which will be accepted internally.

As a general rule, the more support there is for the community internally, the more you can design community-specific metrics.

This also serves as a good rule of thumb. The less internal support exists for the community, the more you need to skew the metrics in favour of value over impact.

 

Using Existing Benchmarks

The first step is to find out if there are already metrics which can be used to benchmark the community against.

These might not be dollar-value metrics. Instead, they might refer to things like member sentiment (or change in sentiment). In this case, you might compare a community by the quality of answers provided by members or the quantity of content created.

There isn’t any shortage of potential impact metrics you can use here. So begin by asking a few key questions for whatever department or purpose(s) your community serves. These questions might include:

  • How are results measured?
  • What metrics are important? And why?
  • How are those metrics calculated?
  • Who gets the metrics and what do they do with them?
  • Is the community likely to be compared against them?

Based on the department your community serves, there are some common metrics you might need to use here. These include

  • Net Promoter Score
  • Customer Satisfaction Score
  • Customer Contact Rate
  • Task Completion Rate
  • Product Adoption Rate
  • No. Bugs Resolved.
  • No Features Implemented

If you adopt one of these metrics for your community, be mindful of the context.

There are many areas where a community will naturally have advantages or disadvantages against other programs.

One client, for example, had terrible NPS scores compared with other customer support channels. The problem wasn’t the community experience. The problem was the community had been positioned as the final destination for people who hadn’t been able to solve their problems through any other channel. These are the same people who were already frustrated before they arrived in the community. Often they simply had problems which couldn’t be solved and had nowhere else to express their frustration.

Comparing the NPS scores of someone in a community vs. someone who gets personalised help on a support call doesn’t make sense unless cost is considered in the conclusion. This is why you might want to compare NPS scores/cost of each contact as a metric rather than just NPS scores.

Never take a single benchmark from another program and apply it to the community. Always consider the unique context and ensure this is reflected in your recommendations.

 

Is Engagement The Goal?

If you don’t have existing benchmarks to compare against, you have the option to design a unique metric for the community. The most common example is to use some measure of engagement.

We’ve talked endlessly about the challenge of using engagement metrics to serve as the north star (I wrote an entire book about it).

Put simply, the problem with using engagement is many people simply don’t value engagement. Even those who just want to see a lot of engagement in a community recognise not all engagement is good (if they disagree, try removing the spam filter).

We’ve worked with dozens of clients over the year who just want to see a highly engaged community. They might not know precisely what success looks like, but they know having a huge number of people highly engaged in an ecosystem they control is critically important.

This is where you want to probe a little deeper to find out exactly what they want to see.

  • What is your definition of community?
  • What kind of metrics do you want to see?
  • How will you know if the community is achieving its goals?
  • What does success or failure look like?

You might not need precise metrics here, but you should be able to get to a business question which you can then translate into more precise metrics.

 

Setting Engagement Metrics

If engagement is the goal, the next step is to translate the broad goals you’ve been given in discussions into more specific metrics.

This is where you need to ‘operationalize’ a goal. This can sound far more complicated than it really is. Essentially, you look for metrics which reflect what stakeholders have said they’re trying to achieve.

For example “I want to see a thriving community with thousands of members reflecting our diverse audience and lots of healthy discussions”

In this example, you can see the number of members, diversity of the audience, and quality of discussion matters. So you might begin testing metrics such as

  • No. active members (1+ posts in the past 30 days).
  • % of members from each core audience group.
  • No. responses to questions.

There aren’t any rigid rules about this process so go with what works best for you. Notice how important it is to pay close attention to language. If the person had referred to thousands of ‘happy members’, we would have also incorporated satisfaction score in there too.

Develop the potential metrics and check internally if they accurately reflect the business question.

Resource: CMX Indispensable Community Talk.

 

Establishing Community-Driven Impact Score

If engagement isn’t a goal (but you know there aren’t existing benchmarks to use), you have the freedom to come up with a better method. This is where we recommend the Community-Driven Impact Score.

This is a simple process where you ask members what the impact is and multiply this by the no. members represented.

For example

  • On a scale of 0 to 10, to what extent did the community influence your likelihood to renew your subscription?

If you have an average score of 8.8 and 10,000 visitors, that’s a score of 88k. Your job is to make that score go up by increasing the number of visitors or improving the impact of community upon members.

In our minds, this is the purest score which reflects the unique impact of community. However, it requires deep internal support and understanding to be fully utilised.

Resource: Setting Up Community-Driven Impact Score.

 

Next Steps

Begin the process of finding a north star metric with the full process in mind. Once you know the process, you can start guiding others through it and find a metric which works for everyone. This includes

1) Outline the process for your colleagues (and bring everyone along on the journey). There isn’t much point in developing a north-star metric if no one else supports it. Instead, if you bring people with you on the journey of discovery you can ideally land upon a metric which works for everyone. Make sure everyone understands the process of finding your north star.

2) Make sure people understand the limitations of metrics. If you need to calculate the dollar value return, make sure everyone understands the limitations of each approach and try to find how other departments have accepted the trade-off to find the right metric.

3) Use existing methods and benchmarks where possible. If you don’t need to show a dollar value, find out if there are existing benchmarks to compare against and if not you can develop community-specific metrics for your organisation. This might mean tracking some measure of engagement or utilising the Community-Driven Impact score.

4) Repeat this process this when things change. This isn’t a one-time process. As stakeholders and corporate strategies change you will likely need to revive and update the process over time. Renew the effort as often as needed and try to expand the value of the community over time.

If you feel you need expert help to develop this process and gain internal support, contact FeverBee.

 

Resources

Article: The Tyranny of Metrics
Article: The Engagement Trap
Article: Don’t Use Bogus Metrics
Template: The Community ROI Flow-Chart
Template: 46 Questions for Uncovering Community Goals
Template: A Framework For Hosting A Successful Workshop To Gain Alignment On Community Goals
Article: Setting Up Community-Driven Impact Score
Talk: CMX Indispensable Community Talk
Book: The Indispensable Community

Become A Community Consultant at FeverBee (North America only)

We’re looking for a community consultant to join the FeverBee team.

If you want to work at the cutting edge of the community industry, help the world’s largest organisations solve complex community challenges, and master a range of new skills, we want to talk to you.

Put simply, our mission is to help great companies build thriving communities.

We’re the consultancy the world’s top brands turn to when they want to improve how they build their communities. Our recent clients include Google, Microsoft, Intel, Apple, Meta, SAP, HP, Esri, Atlassian, Okta, and many more.

We hire passionate, intelligent people, and give them the autonomy to do exceptional work.

What you will do

You will work with clients in a wide range of sectors and industries. Our work is multi-disciplinary and covers corporate strategy, user experience design, data analysis, internal and external communications, and much more. You will be exposed to a wide variety of skills and resources.

Working with our team, you will help shape community strategies at some of the world’s most prominent organizations. This work includes gathering and analysing quantitative and qualitative information, developing and presenting recommendations, and helping implement solutions to our client’s community challenges.

As part of the FeverBee team, you will receive exceptional training as well as frequent coaching and mentoring from our senior team. Additionally, you’ll receive a stipend to undertake training courses, attend events, and purchase relevant books as needed.

 

Key responsibilities include:

  • Undertake user experience research on behalf of clients.
  • Interview community members to understand needs, pain points, and desires.
  • Interview key stakeholders to uncover challenges, understanding, and enthusiasm for community.
  • Evaluate client communities against best-in-class standards.
  • Work with FeverBee’s data analysts to identify key trends, challenges, and value of a community.
  • Develop playbooks, training materials, operational plans, and roadmaps for clients.
  • Use community research to develop and implement best-in-class solutions

 

Requirements

  • 2 to 5 years of professional community experience with hands-on experience using at least one of the following community platforms: Khoros, Salesforce, Discourse, Vanilla, Insided, HigherLogic, or Verint.
  • Located in North America (this is a remote position).
  • Deep understanding and/or past experience in at least one of these fields; design, UI/UX, data analytics, or business strategy.
  • A Passion for analytics and/or statistics.
  • Excellent ability to work in a team environment, and engage effectively with people at all levels in an organization.
  • Ability to communicate complex ideas effectively, both verbally and in writing, in English.

 

Desired strengths

  • Learner – asks thoughtful questions, catches on quickly, interested in people and ideas.
  • Adaptability – receptive to change, flexible, can collaborate with a diverse range of people.
  • Deliberative – exhibits good judgment, identifies risk, and makes sound decisions.

 

Salary

  • $70k to $90k – Depending upon experience.

 

How To Apply

To apply for this role, send an email to [email protected].

Include your résumé and a link to your LinkedIn profile (if you have one).

Most importantly, tell us:

  • Who you are.
  • What you’ve done.
  • Where you’re going.
  • What drives you.
  • What skills you bring to the table.
  • Why you want to do consultancy work.

In short, sell yourself (but don’t tell fibs!)

Applications close on Dec 2, 2022 (but please don’t leave it until the last minute).

EO/EE statement
FeverBee is an equal opportunity employer and prohibits discrimination and harassment of any kind. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Community-Generated Revenue – Maximise Your Community ROI

You might be facing pressure to show your community doesn’t just reduce costs, but directly generates revenue.

In the long term, this pressure may be useful. It’s always a little awkward to ask for more money to support a program designed to reduce costs.

But in the short term, it presents a challenge; how do you generate revenue through community?

This is a question we should welcome and definitively answer. Cost savings are limited by definition. However, there is no cap on the ability to generate more revenue.

Better yet, the more revenue your community generates, the more internal stakeholder support you’re likely to get.

 

How Can Brand Communities Generate Revenue

Brand communities typically generate revenue in three core ways.

These include:

  • Retention (keeping customers you have). Brand communities directly improve the retention rate. This includes improving trial completion rates, increasing the lifetime value of customers, or improving retention of specific niches.
  • Direct sales (selling your members). Communities create opportunities to sell new products to existing members and drive greater product adoption.
  • New customers (attracting new business). Communities attract new customers (and audiences) to the brand’s products and services.

Each method has clear drawbacks. Retention is hard to prove, direct selling can be irritating to members who didn’t ask for it, and it’s not easy to attract customers of rival brands to a community. Yet it’s possible to navigate through each approach and harness the community to show great results.

 

Driving Product Retention Through Community

We all know brand communities can increase customer loyalty (i.e. retention).

But this isn’t achieved simply through members engaging with one another. Often the most engaged members are the most loyal customers, to begin with.

Instead, we have discovered there are very specific mechanisms that increase retention. You can see these below:

How do communities drive ROI

We won’t cover every activity, but retention is achieved when members experience:

  1. Initial success using the products as quickly as possible.
  2. Learning how to get the most value from the products.
  3. Satisfaction with the brand experience.

Each of these, as you can see above, can be influenced by things you directly control.

Here are a few things to consider:

Set Up Groups For Product Newcomers

It’s common to set up a group for newcomers to the community. This isn’t a terrible idea, but it’s not the best idea either. People don’t want to learn about the brand community, they want to learn how to get started with the product. It’s far better to set up groups for people who are new to the product. This is a place where they can be encouraged to ask as many beginner-level questions as possible and get mentored by others.

This works best when you collaborate with sales and success teams who will guide new customers into newcomer groups. If members are on a trial period, it’s critical to guide members into groups where they can experience results as quickly as possible.

You should ask members to frequently share updates and progress, solicit questions on areas which members often don’t understand, and have top community mentors who can provide immediate-hand help to resolve any issues.

Measurement: Track the % of newcomers who complete the trial in a newcomer group compared with those who don’t.

 

Prioritize Questions Of Trialists

You can also improve the retention rate of trialists by prioritizing their questions. This is the audience most likely to give up if they experience frustration. Every hour they have to wait for a response to a problem, their frustration increases.

Set up an automated or manual system to highlight questions from community newcomers (first 30 days) to a superuser group or internal Slack channel. Work with your top members to prioritize responding to these questions and guiding members to provide the best possible responses.

Measurement: Track the satisfaction rate of trialists invited to the community vs. those who aren’t.

 

Build A Library of Great Examples

In our experience, members love seeing great case studies, examples, and breakdowns showing the best (and most innovative) ways to use a product. They use these to guide their own work and get more value from using the product.

Instead of badgering members to share their best advice in wikis (or tribal knowledge base), it’s far, far, better to encourage members to share how they’re using your products and services. Especially if they have a unique approach to it.

A single great case study or breakdown provided by a member of the community is worth dozens of responses. It’s one of the most valuable contributions members can make to your community. You can use a tribal knowledge base tool, interview members, or simply invite members to submit articles for the blog or newsletter.

The key to making this work is ensuring the reward vs. effort equation is in their favour.

Measurement: Regression analysis showing the relationship between article views and retention rate * customer lifetime value.

Resource: How To Get Experts To Contribute To Your Community

 

Upgrade Members Near Renewal Time

If you are managing a community where members renew their subscriptions each year, make sure you provide increased value to members closer to this time.

Create an automated system where a few months before a member’s renewal is due, invite them to a private group of peers to celebrate their one-year anniversary. This should be a group at a more advanced level and ideally where they can find peers to help them with the next stage of their journey.

This provides unique value to members and targets this value at the moment when members might begin considering whether or not to renew their subscriptions.

Measurement: Track the renewal rate of members who receive these notifications vs. those who aren’t.

 

Direct Sales To Community Members

The idea of selling things to community members feels like an anathema to many of us. This is often the result of painful internal discussions where we fight to keep the hounds of salespeople out of our community.

However, if you’re in business, you’re always in sales in some capacity. Don’t shirk from this opportunity, embrace how you can showcase products in a manner which feels authentic to you and your community (e.g through UGC videos, user group meetups, etc). The key is to ensure you’re selling what members truly need and check the sales process adds value to members.

When done well, this is the most direct path to drive immediate value and show the quickest results. You don’t need fancy math to show direct value generated from selling to members.

 

Premium Membership

The easiest option is to charge a fee to be a part of a community (or to access unique areas/features of the community).

The downside of this approach is it immediately limits the size and level of engagement within the community. Yet, it may be possible to create unique user groups, membership tiers, or features that members are happy to pay for. Sometimes this can be wrapped into a broader support package.

You might, for example, pay a small fee for top members to share their best expertise in a private group that only paying members can access. Or offer members unique customisation opportunities to engage in a certain way.

 

Events

Perhaps the most common approach is to charge members a fee to attend community-hosted events and activities. When this is done well, it can be a tremendous revenue driver. You can attract sponsors, new customers, and members paying hundreds, even thousands, of dollars to attend.

The downside is events are cost-heavy. Many community events are loss leaders to attract new customers and build goodwill. While there are great examples of community events being spun into their own business unit, the scale required to make this work (and the variable cost model) makes it a challenge.

 

Courses

A far better match for most organizations is to develop training courses members can take.

If this is structured as an on-demand course, you can offer members courses that may be related to your product but not about your products.

You might not even need to create all the content yourself. Some organizations, like CXL (a client), pay top industry professionals a fixed fee and package access to all courses as part of a unique membership tier.

Courses could also be sold individually. You can reach out to three to four relevant industry experts and pay them a fee to create a course. You then sell the course to community members (or with a special discount to members).

 

Affiliate and Partner Sales

Perhaps the least common route in a brand community is affiliate sales.

When you have access to a large audience, there will be plenty of organizations that will pay a fee to reach them. Sometimes this is directly achieved through advertising. But few brands want advertising in their brand community.

But advertising is just one of many approaches. I know one organization in the pharmaceutical sector which generates millions of dollars a year by recruiting community members to participate in focus groups for a partner.

In this approach, you find partners in your industry who would like to reach your audience. You work with them to offer a strong discount (i.e. the unique value to members), and you gain a percentage figure for every product sold. Be mindful that any promotion is limited in frequency and offers tremendous value to members.

If members begin to express dissatisfaction, this approach should be discontinued.

 

Generate Business From The Community

Your brand community can also generate new business.

If the previous section explained how to sell new products to your existing customers, this section describes how to attract new customers to your existing products.

Of the three approaches, this is by far the most popular with organizations in today’s economic climate.

Brand communities can attract new customers in two primary ways. They can attract new leads and increase the likelihood of existing leads becoming customers.

 

Generating Leads

Generating leads is probably the easiest activity to directly show new business.

Theoretically, every registration is a lead. The problem for most brand communities is the majority of members are already customers of the brand.

The challenge is to attract people who aren’t already customers to join and participate. This raises an obvious question (one which many people launching a new community fail to answer); why would someone who doesn’t use our products join our community?

 

1) Create A ‘Thought Leadership’ Zone

A common answer to the question above is to create a community about the topic, not the brand.

In theory, this can work well (and there are plenty of great examples).

In practice, it’s hard to do. Most efforts struggle. Members don’t have the same urgency to engage and participate. And the people who have the best expertise to share often want to do it in channels where they can build a following.

To do this well, it’s important to make sure you recruit experts early and ensure they’re getting what they want from the experience.

Engaging Experts in your communityIn this approach, you can open up a section of the community (or create a distinct area) for industry-focused topics and, as long as you are transparent, you can attract leads through registrations in the community.

Resource: Why You Should Build A Thought Leadership Community

 

2) Create Gated Resources

A simpler approach than creating an entire community to attract leads is to create resources and activities non-customers would be eager to submit their email addresses to access.

In this approach, you can collaborate with your members to create shared resources (or host events). These can then be promoted to non-members through advertising, social media, and other channels. Every registration can theoretically be counted as a lead.

 

3) Lead-Scoring Through Community Behavior

A powerful, but technically complex, approach is to identify leads from community behavior. In this approach, you might identify leads (or, more likely, upsell opportunities), by seeing how members engage in the community.

This can occur through lead scoring. This means specific behavior in the community is assigned a score. When a member achieved a certain score, they’re considered a lead and receive an invite to the next stage of the process.

A less complex version of this is to manually identify leads. For example, if you notice members discussing a problem one of your products can solve, you might flag this to a salesperson. We used to do this in the FeverBee Experts community.

 

4) Drive Traffic To The Website

A final approach is to optimize the community for search and leverage the community to maximize the traffic it drives to your primary company website.

There are plenty of ways you can optimize a community for search. This includes archiving old and duplicate content, optimizing titles for search, and ensuring most questions receive an accepted solution.

If you get to a point where your community is the secondary driver of traffic to your company homepage, you’re in a great position.

Who would want to reduce support for the community if it has a serious negative impact on web traffic?

 

Lead Conversion

Communities can also play a unique role in improving lead conversion. Sometimes it happens naturally. People want to get backchannel feedback or insights on the product/service they want to purchase. The community lets people ask questions and get help from others who have been in their position. In a branded community, they can also see how they will be supported as an admin and/or user of that product.

You can also stimulate improved lead conversion several mechanisms.

 

1) Showing Community Content On Product Pages.

You can use community-generated content on product pages and throughout the company website. This lets you use the natural social proof created by your community within the product pages where they will have maximum impact

Sephora (a client) is a good example of this.

Sephora Product Community

Looks, reviews, and discussions sourced in the community appear on product pages – this directly increases conversion.

Resource: Community Content Creates Trusted Product Pages

 

2) Surfacing Community Data in Presentations.

Having a thriving community should be a powerful reason for a potential customer to select your product over another.

If you have a community of 30k members who answer hundreds of questions a day and your competitors don’t, that’s a major differentiator.

The problem is the community data often isn’t communicated to prospects – at least not in a decisive way. The size and scale of the community should be prominently mentioned in sales materials as a major strategic differentiator and a reason to purchase.

Make sure the sales team is provided with good, useful, data they can use to highlight the unique value of the community. You can find out exactly what statistics might be most impressive and provide them with data and examples which might be a great fit. If you can build a list of great examples (discussions, case studies), this is a huge benefit.

Resource: Competitive Benchmarking

 

3) Testimonials, Reviews, and Case studies.

Your community is a fantastic tool to generate testimonials, reviews, and case studies which can be used in sales material and featured on comparison sites to directly attract more sales.

If you source 50 reviews for a comparison site from your community, it’s likely you will instantly jump to number one within that category. For an enterprise product, this can generate millions of dollars in sales. It’s one of the quickest possible wins.

To make this work, you need to make this part of the community management process. Regularly invite top members to create case studies, share testimonials, and leave reviews on relevant sites.

Make it as easy as possible by creating a template they can use. The more reviews and testimonials you can source from your members, the more leads you’re likely to convert. You can begin by asking your sales team which kind of testimonials would be most impactful and work with top members to do precisely that.

(p.s. make sure you have a large group of community members willing to serve as reference calls for prospects as well).

Resource: How Quickbase Became No. 1 In Their Category

 

4) Community-Recommended Products

If you’re selling multiple products to members, encourage members to share their product stack / recommended products with one another. This is especially useful for retail brands.

Better yet, show related products next to relevant community discussions and track purchases from these discussions. Having a list of community products recommended by members helps reduce the perceived risk and may increase sales.

 

5) Create Product Hubs / Pages Within A Community

You might also consider creating a product hub or page within a community which pulls together community discussions, one-pagers and data-sheets, feature requests/ideas, videos around key products. This is a highlighted, curated, collection of community and company products all within the same place.

 

Create Community-Generated Revenue

Don’t hide from the revenue discussion. If you have a successful brand community, it’s likely already generating revenue or there are a few small changes you can implement to drive tremendous revenue.

Try a variety of things and uncover which works best for your audiences (colleagues and members). Not every activity will hit, but you will be able to find a combination which works to turn your community into a revenue-generating machine. Make sure you track the results and consistently show how much revenue the community is generating.

It’s one thing to say a company should invest more to reduce costs, it’s a lot easier to show a company they should invest more to drive more revenue.

Feel free to contact us for consultancy help. We’ve helped clients generate millions of dollars from their community efforts. We would love to help you too.

 

Resources

Template: How Do Communities Drive ROI?
Guide: Measuring The ROI of Online Communities
Book: The Indispensable Community
Article: The Million Dollar Community Page
Article: How To Get Experts To Contribute To Your Community
Article: Why You Should Build A Thought Leadership Community
Article: How To Get Experts To Contribute To Your Community
Article: Community Content Creates Trusted Product Pages
Article: How Quickbase Became Number 1 In Their Category
Community: The CXL Playbook Community

Designing Your Community Data System – From Basic To Elite

Many of us haven’t really scratched the surface of what’s possible with our data.

If you want to see what’s possible, I recommend you watch this talk if you haven’t already.

You can do far more with your data than you probably realise.

Note: You should also sign up for our workshop on building community data systems on Nov 2.

Yet just knowing what’s possible doesn’t help us achieve our goals. We also need the knowledge to jump from where are today to get to where we need to go. And that destination is a place where we finally have the data (in the manner we need it) to make the right decisions for our community.

 

Benchmark Your Current Data System

If you’re trying to jump from a novice to an expert overnight, you’re setting yourself up for disappointment.

It’s far better to figure out where you are today and track your progress over time.

community data benchmarks

As you can see above, we benchmark community data skills along the four levels.

To summarise briefly:

  • Basic. Tracking changes in community engagement.
  • Advanced. Tracking what happened and why it happened by using external data.
  • Professional. Evaluating community impact and making data-driven improvements.
  • Elite. Automating the entire process.

These benchmarks themselves are comprised of five attributes:

  • Metrics. Which metrics do you measure grow and expand over time?
  • Extraction. How do you gather the data from different sources?
  • Analysis. Which analyses do you undertake in your community?
  • Summary. How do you summarise and visualise the data?
  • Utility. How do you utilise the data in your strategy?

Before we begin, take a second to benchmark your current level today. You might be farther along in some attributes than others.

If you’re like most people, you’re at the beginner or advanced level. This is good – it means you can make a lot of improvement in a relatively short amount of time!

Resource: Community Data Benchmarks

 

The Basic Level

If you’re relying upon the insights that come up with your community platform, you’re at the basic level.

Almost all of us begin by using our community platforms to track engagement. Yet even within this level, there is a huge variation in the data you have.

The quantity and quality of insights provided by inexpensive tools like Facebook, Circle, and Tribe are very different from those provided by premium enterprise platforms.

Generally speaking, the more you spend the better insights you should have.

At the basic level, you can usually answer the following questions:

  • Is engagement heading in the right direction?
  • Who are the most active members of the community?
  • What are the popular discussions in the community?

Overall, you get a simple snapshot which reflects aggregated data for a given time period in a community. Typically at this level, you’re simply gathering community data and sending screenshots or visuals to colleagues.

However, there are critical insights you won’t have. For example:

  • What’s causing the changes in engagement (and what to optimise for)?
  • Are changing because of your actions or things happening outside of the community?
  • Where are people dropping out of the community journey and why?
  • Is the community having an impact on key business metrics?

As your community expands (or your ambitions expand), not having this data starts to become a bigger and bigger problem.

Sooner or later, you need to move on to a more advanced level of insights.

 

The Advanced Level

The advanced level is when you begin to get serious about community data.

At this level, you’re not just concerned about what happened, you’re concerned about why it happened. The why is what lets you improve your community efforts. This is the foundation stone upon which you build a data-driven community strategy.

e.g. If you know [x] is causing a positive outcome, you probably want to do more of [x]. 

The key difference at this level is you need to combine multiple datasets. You’re still working with aggregated data (i.e. behavior is combined into a metric rather than separated by members), but you need to wrangle a few other sources of data together.

 

Step One: Gather Environment-Level Data

At this stage, gather data on what’s happening outside of the community to see how it impacts things within the community. We could easily write a series of articles about the precise metrics to measure, for now, I’ll recommend our ROI guide and this guide

I would recommend getting monthly time-series data to reveal:

  • No. visitors to the company website. (Google Analytics, Adobe Analytics, Amplitude Analytics etc..).
  • No. unique, new, visitors to the company website (Google Analytics, Adobe Analytics, Amplitude Analytics etc..)
  • No. search visitors to the company website (Unique visitors – source: Search).
  • No. customers/users (sales data/usage data – usually upon request)
  • No. new customers/product activations (sales data).
  • No. support tickets/customer calls per month (help desk software/customer support software – usually upon request).
  • Any proxy metrics for no. developers, admins, users etc…

Download this data from each source individually. Remember to keep rows as months (use transpose if they’re in the wrong format) and ensure it matches up on a monthly time-series basis stretching back up to 36 months (any longer than this and the data is less relevant).

You can usually download these metrics individually from each source. However, it might take a little effort to clean the data so you can create a single spreadsheet. The spreadsheet should have each attribute above as a column and each month should be a row.

Note: It’s difficult to download monthly time-series data from some platforms (e.g. Adobe Analytics). You might need to download data from each month at a time. This is also true of some community platforms. This is painful and takes a little time – but you only need to do it once (thereafter you’re updating the data each month).

 

Step Two: Setup and/or Collect Qualitative Performance Data

Quantitative data can reveal plenty of useful insights, but it doesn’t show whether members are becoming more or less satisfied with the community. This is why it’s also good to have a measure of member satisfaction you can track over time.

This can be done in three ways.

  1. An annual survey. You can set up an annual survey of up to 10 questions to get an ongoing measure of how members feel about the community. This will go out to all members (ideally using the same questions each year).
  2. A quarterly survey. You can set up a survey which goes out to a rotating sample of members at random. This makes it easier to track data over time but is prone to strong fluctuations if you don’t get 300 responses.
  3. A poll. A poll is an ideal solution to track member satisfaction. This appears to members as a pop-up in the community and asks members to rate their satisfaction, whether they got the answer or the overall impact of the community. These results can be aggregated into a summary score over time. I’d strongly recommend setting up the community-driven impact score system at this level.

You can set this up in your platform toll or using Hotjar, Qualtrics, Typeform, SurveyMonkey, or any other tools which suit your goals.

Ideally, you should combine this data with your community data on a monthly time-series basis.

If you can’t get data monthly (it’s hard to do this for surveys unless you sample a rotating group of members), simply share the annual data each month for now.

 

Step Three: Analysing Community Performance

Once you’ve got your dataset, you should be able to analyse community performance.

You can see the kinds of questions you can answer below (but, be mindful, this isn’t a definitive list).

Question:
What are the trends we need to adapt to?
Data
What content / categories / tags are rising or falling in popularity (use % from normalised baseline).Aggregate group activity by category.
What areas, features, sections of community are rising or falling in popularity?Aggregated visitor data from different sections of the community.
Is there any major shift in locations people are coming from?Google analytics - top 10 locations - 100% bar chart with trendines.
Are the type of users changing (more devs, admins etc..).Member profile data (or synched customer data).
Any specific changes between types of visitors?Google Analytics data on devices, regions, languages etc?

To perform this analysis, you might also use your current variables to create new metrics.

For example, you might create metrics like posts per active member, visitor conversion rate etc…You can find an example of what we often look at in this sheet.

You can now create graphs to help you answer questions like:

  • Is the community doing well in attracting attention and acquiring newcomers? (i.e. attracting the expected number of visitors, newcomers, and questions)
  • Does the community have the right number of superusers / is the quality of superusers good enough? (i.e. are the majority of questions being resolved/answered as the community grows?)
  • Are customers more or less likely to turn to the community for support? (i.e. what is the relationship between support tickets vs. community questions?)

You don’t need to use these questions verbatim. Come up with your own questions if you need them. What you should be able to determine though is how well the community is performing and why.

Note: Be mindful that this is always correlational data. This doesn’t make the data bogus, but it should be treated as a relationship which could be explored further.

 

Step Four: Using the data

If you have the expertise, you can run a regression analysis to show the strength of the relationship numerically. But this isn’t always easy to interpret.

Often simply looking at the data on a chart will help you explain what’s happening. 

It’s good to highlight any significant changes within the organisation which might’ve taken place during this time frame to see how they impacted the community (e.g. new product launches, website redesign, Black Friday etc…). 

You can overlay these with the data itself to see if there was any major impact.

You should be able to tell a compelling story here which explains:

  • What’s happened so far?
  • Why did those things happen? 
  • What’s likely to happen in the future (the forecast function is worth learning)?
  • What should you do more of, less of, and change to improve results?

Resource: Setting Up The Community-Driven Impact Score System.

 

The Professional Level

At the professional level, two things change.

  1. You look at member data. Instead of only tracking outcomes by month, you track outcomes by member. For example, you might see if members who have been invited into a newcomer group had a higher product trial completion rate than those who didn’t.
  2. You integrate databases. Instead of getting a single metric from each source per month (i.e. 16,434 customer calls), you get the list of who made the calls and when. You need to combine data from several databases to show the impact of community.

This lets you do a far deeper analysis to determine what is and isn’t working in your community. You can see exactly which activities (or member attributes) had a meaningful impact and which didn’t.

As this example dataset shows, once you have data at the member level the quality of analyses you can undertake improves considerably.

 

Step 1: Identifying sources of member data. 

Member-level analysis requires you to build a database of individuals where each row (observation) is a member rather than a month. This database needs to combine records from several sources.

These sources might include:

  • The community platform.
  • The customer relationship management system (CRM).
  • The mailing list provider (see who did and didn’t open emails).
  • The learning management system (LMS) (see who completed training courses).
  • The call or help-desk system (see how filed tickets).
  • Social media platforms (community intelligence tools).
  • The event /usergroup hosting software (see who attended events).
  • Etc…

In larger organisations, it’s likely there is already some database which integrates a lot of these things. You might simply be able to add community data to it. The challenge is finding out where it is, who owns it, and in what format it’s in.

It’s also common at this stage to use community intelligence tools like CommonRoom and Orbit to create a more detailed picture of member behaviours from a variety of different channels (i.e. you can learn about member engagements on Reddit, Discord, Slack, and other social media channels).

 

Step 2: Extract and combine the data

Sometimes you get lucky and the platforms you’re using have a connector which lets you easily combine data (and keep it updated). It’s far more common, however, that you will need to pull together the different datasets yourself.

This might be by extracting the data from the available APIs, or it might mean manually downloading and combining the data from each source individually. The challenge is often matching up records of different individuals across platforms. If members don’t use a consistent email address, this becomes tricky (but not impossible) to do.

Even if you get the data, the quality of it might vary considerably. API data can be in the form of raw server logs which you then need to process to be useful.

Another challenge is it’s very likely you will be working with datasets which are too large to process on your personal laptop – so you might need to consider virtual solutions for collecting, processing, and securely storing the data (data security is going to be a critical consideration).

As a general principle, cleaning and transforming the data usually takes 80% of the entire scope of the project.

 

Step 3: Analysing the data

Once you have data on individuals, you can still do some really exciting things.

This includes

  • A sequential analysis to determine the order of member behaviors and community participation (i.e. see what came first).
  • Survival analysis to see how retention rate varies by month and member segment.
  • Analyse superuser performance by accepted solution rate.
  • Segment the community by category and see which areas need more attention.
  • See how product usage varies by community participation and types of participation.
  • Identify attributes which cause members to stay for longer.
  • Identify unique clusters of members by community behaviours and build distinct mailing lists of them.
  • Develop detailed journeys of each member.

You should be able to build a model which predicts the outcomes you want to gain from the community and then gain insights into what you need to do to drive those outcomes.

However, this requires expertise in statistics. If you don’t have expertise in this, we can set up this system for you.

Note: you have to be really, really, precise about the definition of any metric you use here. For example, if you want to know the impact of the community on customer retention. You have to define what metrics best represent each of those terms. Two different people can define the terms in two very different ways and come to different conclusions.

 

Step 4: Summarise data in a custom dashboard

Once you have a clean dataset, you can develop a custom dashboard in Tableau, PowerBI, or any tool you’re familiar with. Again, this requires some technical knowledge. But here we would recommend having colour-coded dashboards which help you make the critical decisions you need to make about your community.

Common metrics here would include:

  • Activity-level metrics (no. posts, active members, newcomer retention, and no. active MVPs).
  • Program-level metrics (% questions with a response, average time to first response, accepted solution rate).
  • Satisfaction-level metrics (from polls/surveys).
  • Impact-level metrics (impact upon customer retention, utilisation, or community-driven impact score etc..)

You should know the primary drivers of these metrics and be able to produce performance-based forecasts.

This dashboard should have pre-determined success/failure metrics which identify whether unique community programs succeeded or not. Those programs which succeeded should be provided with additional resources and those which didn’t succeed should be altered or have fewer resources in the future.

From this, everyone can determine both how the community is doing today and what to focus on next.

Resource: Setting Targets and Building A Community Dashboard

Resource: The Dream Community Dataset

 

The Elite Level

The big difference between professional and elite is the automated data pipeline.

A simple overview of a data pipeline is shown via Newsela below:

(p.s. you should watch the full presentation hosted on the Verint/Telligent Community)

At the elite level, you’re not manually pulling and analysing data. It’s not a monthly task you have to (painfully) endure. Instead, it’s a fully automated pipeline of fresh insights.

Typically, data is extracted via APIs (and direct connectors), cleaned, transformed, and stored within a data warehouse. From there a business intelligence or data visualisation tool is used to explore the data. You can provide different colleagues with different levels of access to the data depending on their needs.

This is infinitely more scalable and it gives you regularly updated data (to the day if needed) which others can explore. For larger communities, it’s beneficial to get real-time insights rather than wait until the end of the month to see what’s happening.

The downside is this is a project which requires significant investment. It requires significant resources, expertise, and time. You also need to account for maintenance costs (data pipelines frequently break when platforms are updated or tweaked ).

Before beginning this journey, you need to carefully consider issues such as data quality, integrity, governance, privacy, and security.

The steps you take might vary, but this is what we would usually recommend:

 

Step 1: Create a steering group to define the outcome

Every process is different, but it’s useful to begin by bringing the right people together to start. I’ve lost track of how many data projects stumble because of issues which weren’t foreseen at the time.

The steering group varies but often includes a combination of:

  • Community team
  • Community vendor
  • Company software engineers
  • Customer loyalty/success
  • Marketing
  • Sales
  • Legal
  • IT (esp. Data security-minded folks).
  • Data-specific people

Essentially anyone with any sort of influence or interest in data from the sources you’re likely to collect data from should be included in this process. Your primary responsibility at this point is to learn.

Always begin this process by bringing the right stakeholders together to gather requirements, identify issues and challenges, and understand their needs from the data. It’s likely others want to ensure key outcomes are considered too.

 

Step 2: Decide who will be doing the implementation

The next step is to define the right architecture for your data system.

If you don’t have a data engineering background, please don’t try to do this yourself. Selecting the right architecture and technology requires considerable knowledge of the competing options and a deep understanding of your organization’s own needs and requirements.

In the past, we’ve seen a community data pipeline shut down after two months because the organisation’s data technology stack was moving to a different set of tools.

The earliest decision you need to make is whether you have the resources to do this internally or not. This is whether consultancies can help. You need someone to work with your team to define the requirements in architecture and help select the right technology stack.

 

Step 3: Implement the solution

This is a bigger topic than we will cover here. Perhaps the most important difference in this process is you will first import data from a variety of sources into a data warehouse (Snowflake, BigQuery, Redshift, etc…).

In some cases, this might involve writing a custom webhook to import the data and using separate data orchestration tools. You will also need a data analytics tool which integrates with your data warehouse.

We’re not going to cover the process of building the full data pipeline here. There are far better resources than us to learn about data pipelines. 

Be mindful that this is an intensive process – one that often involves considerable trial and error. This will usually take several months to complete. 

 

Step 4: Training stakeholders to use the data

The final step in the journey is to develop a process for persuading and training stakeholders to use the data to support their work. This applies to both your immediate team and to a broader set of community stakeholders. 

This involves both a combination of emotive-driven storytelling and training workshops where you can guide stakeholders through the process of extracting the data they need. 

I’d recommend having three things in place.

  • A simple roadshow deck explaining the remarkable value of using the tool. This should cover the basics of community, the type of insights people can get, and how to explore the data. 
  • A regular workshop (hosted every 3 to 6 months) for stakeholders can attend where you will help them access and explore the community data. This lets them run analyses themselves, address questions and, most importantly, get time on people’s calendars for the community. 
  • A short guide to support colleagues getting set up on the data visualisation platform – included with some basic do’s and don’t’s. At a minimum, stakeholders should know how to run basic queries and explore the data to find useful insights. 

Resource: How Newsela Build Their Data Pipeline

Resource: Fundamentals of Data Engineering

Resource: What Is A Data Pipeline?

 

Next Steps

If you’ve found this guide useful and want to explore it further, I’d strongly recommend attending our community data workshop on November 2nd.

Ultimately, if you want your community taken seriously you have to take your community data seriously.

Community platforms can show you if engagement is heading in the right direction and who the most active members are, but they’re never going to be able to do much more than that. They can’t answer the critical questions you and your colleagues have. They can’t show a meaningful impact caused by the community.

Many of us need to tread beyond our comfort zone to get the data we need. You don’t need to be a technical person, but you do need to know the process and how to advance to each level. You’re also likely to need outside help.

Begin with the questions you want to answer. You can get a good list of these from speaking with your stakeholders. Refine these questions into specific metrics and start identifying the source of each of these metrics. Be as specific as possible.

Get the environmental data you need and start combining it with community data. Once you can do that, get the member-level data from different databases into a single destination. Now you can consider building an automated system for the whole process.

 

Resources

Community-Driven Impact – A Better Way To Prove The Value of Community

Back in 2018, I stood on stage at the CMX Summit and asked the audience whether they believed it was critical to prove the ROI of their community.

About half the hands in the room shot up.

Excellent!

Next, I asked how many of them could prove the ROI of their community.

The outstretched arms began to wilt. Within seconds, just half a dozen hands remained raised in the air amongst a sea of glum-looking faces.

Now it was time for the kicker.

“If most of you believe it’s critical to measure the ROI of community, but almost no one can do it – how critical can it really be?”

I mean, if it was really critical – life and death critical – wouldn’t anyone who couldn’t prove the ROI be out of work by now?

Either we were all on the slaughterhouse conveyor belt waiting for our turn to get the chop, or we were all very much mistaken about ROI.

Maybe measuring ROI isn’t as critical as we think – at least not in the way we imagine ROI.

Perhaps there’s a better way?

A week ago I hosted the Community-Driven Impact webinar to share a better approach.

It seemed to resonate so I’ll share the highlights here

What Do We Measure If Not ROI?

No one is proclaiming we should be free from the shackles of measurement.

Measurement is ultimately about having the right data to make good decisions.

At the ROI level, these are decisions about whether community programs should have more or less funding.

If we don’t measure anything, we can’t prove anything (i.e. no resources).

We need measurement. We need to be held accountable for results.

Most importantly, we need measurement to show the incredible value of the work we do.

The problem is ROI is a 100+-year-old efficiency ratio from a bygone era.

ROI was developed in a less complex world of tangible products and slower decision-making. It made sense to use ROI when considering major capital investments (‘should we build a new factory?’), it’s less effective in the intangible, service-driven, rapid-decision-making era of today.

It’s certainly possible to measure the ROI of communities, but it’s horribly complex, takes a huge amount of time, and the result is rarely as accurate or as persuasive as you might think.

Most people, when they refer to ROI, don’t mean the percentage return for every dollar spent. What they mean is what is the benefit of this?

Unless you sell membership to the community, your community is several layers removed from profit-level metrics. It doesn’t make sense to use profit-level metrics to evaluate the community.

My argument is twofold.

1) We should be measuring outcomes the community can and does directly influence.

2) We should be measuring these outcomes using a methodology developed precisely for the community.

This is why we’ve recently begun using a new score to measure community.

It’s a score I want to introduce to you today.

We call it Community-Driven Impact.

 

The Problem With Current Measurements

The problem with pretty much every community value metric is it’s not designed for community.

Every method, as you can see below, trades precision for simplicity and must choose between causation and correlation. Neither is close to a great outcome.

methods for measuring community value

Controlled experiments can show great results, but are incredibly difficult and costly to do.

Comparing members to non-members helps set priorities, but doesn’t show causation.

Call deflection is horribly imprecise. Estimates for ‘visitors to deflected tickets’ range from 1 in 20 to 1 in 500. No one really knows if a visitor to an answer received the answer to their question or not. Surveys can help but suffer from a sampling bias.

NPS can show a change in attitudes, but it’s an advocacy score. You might help a member resolve a problem but that doesn’t make them more likely to recommend a brand to friends (honestly, how often do you really recommend any brand to friends?).

What we need is a different measure. A measure which directly shows the unique impact of the community. A measure which colleagues will care about. A measure which is easy to set up and easier to understand.

This is where Community-Driven Impact matters.

 

Introducing Community-Driven Impact

Community-Driven Impact is a metric which combines the two key things people really want to know about your community.

1) How impact does the community have upon visitors? (Average Member Impact)

2) How many people visit the community? (reach)

The method multiplies the reach by Average Member Impact to get a Community-Driven Impact score.

Reach is relatively simple to calculate. It’s simply the number of people who visit the community in any given time frame.

You can set a cut-off for this if you like (i.e. skip visits of less than 30 seconds), but total reach is recommended.

The harder part is calculating the Average Member Impact.

 

Average Member Impact (AMI)

The Average Member Impact is the average impact directly caused by the community on each individual visitor.

 

How To Measure Average Member Impact

The simple method is to adapt the Net Promoter Score question for a community-specific purpose.

We simply ask our members what they felt the impact was.

To put the question in the simplest terms, we ask ‘How has the community impacted [outcome]?’

Behavioral Impact Score Question

You can adapt this question for almost any desired outcome. For example:

How has the [brand] community impacted your ability to achieve [outcome]?

How has the [brand] community influenced your likelihood to utilise [features of product]?

How has the [brand] community influenced your [results of product]?

How did [brand] community influence your decision to [purchase item]

How has the community affected your satisfaction with ?

You can even adapt this question to metrics like call deflection

“Approximately how many times has the community saved you from contacting customer support?”

The magic of this approach is the question is incredibly flexible, it lets members make the causation, and it’s easy to get good data.

You can collect this data by polls or surveys each month/quarter and average the results over time as you see in the chart below:

average member impact graph

How Accurate Is This?

I can hear the howls of criticism already.

“But members don’t give accurate answers to survey questions!”

“Members might have different interpretations of the question!”

“You might suffer from a self-response bias!”

“What if members can’t remember or recall their past behaviour?”

These are valid criticisms. The results won’t be 100% accurate.

But here’s a sneaky secret; neither are most of the other metrics you’re looking at.

Every data point you ever look at is based upon a wide number of assumptions enslaved to a large number of biases. The key is to balance practicality against bias to find the optimal outcome.

The main difference in this approach is we’re not using a black box of complex formulas to show the impact of community. We’re using a completely transparent method to show the impact of community using a measure designed for community.

It’s impossible to criticise this method while still supporting NPS or CSAT as a valid measure of success.

The AMI question gets people to directly connect their community experience to the desired outcome and make an estimate. It takes the burden of causation away from us and lets members highlight what (if any) impact the community has had on their behavior.

The problem is people are generally not great at explaining what drove their behaviour. For example, people claim advertising doesn’t affect them but they keep buying from the biggest advertisers.

This doesn’t invalidate behavior surveys, it instead shows that all behavioral surveys entail a certain level of uncertainty. We should recognise this uncertainty and reference it in our summaries.

The major benefit of calculating Average Member Impact is it doesn’t require us to undertake statistical voodoo to connect the community to the result, it’s a direct question asking for a direct, thoughtful, response. To assume that members can’t understand that is to think extremely poorly of our members.

 

Calculating Community-Driven Impact

Tracking your Average Member Impact is an excellent way to escape the engagement trap (the trap of being measured by the level of engagement).

If you can show more engagement reduces your Average Member Impact score, it doesn’t make sense to continue increasing engagement.

At least in theory it doesn’t.

But there is an obvious problem here.

It doesn’t matter if your Average Member Impact is really high if your reach is really low.

Few brands are going to be impressed if you build a group of 10 really happy members. That’s too many resources devoted to too few people. The level of engagement isn’t the critical number, but it does matter.

Your boss isn’t mistaken for wanting engagement to go up. Where they go wrong is wanting this number to go up without knowing whether increasing engagement impacts impact.

It equally doesn’t make sense to have a community of 100,000 members if the community has no impact on them (i.e. has an Average Member Impact score of zero or below). That’s just a waste of resources.

To balance engagement against impact, we need to calculate Community-Driven Impact.

 

Community-Driven Impact

The Community-Driven Impact score is your Average Member Impact score multiplied by the total number of visitors each month.

In short, it measures how many people the community reaches and the impact the community has upon them.

For example:

  • An AMI score of 2.7 with 10,000 monthly visitors is a Community-Driven Impact score of 27,000.
  • An AMI score of 4.1 with 7,000 monthly visitors is a Community-Driven Impact score of 28,700.

The above example shows the importance of balancing the two metrics against each other. A smaller community with a higher impact is as valuable as a larger community with a smaller impact.

It’s ok if either metric declines as long as the Community-Driven Impact score remains consistent (or increases).

Ultimately, the CDI score is the one which really matters. It combines the value of both reach and impact so people can evaluate community success over time as you can see in the graph below.

community driven impact score graph

Above you can see how both changes in reach and Average Member Impact influence the Community-Driven Impact (CDI) score.

We can see in this example the community is delivering more value over time.

 

This Data Reveals Powerful Insights

These metrics also help you analyse what’s happening in the community.

For example, a client was seeing a long-term decline in monthly visitors but an increase in Average Member Impact.

When we began segmenting the AMI score by member type, we noticed a strong relationship between member tenure and the AMI score.

The AMI score was increasing because people with a lower AMI were leaving the community.

We discovered the community was increasingly catering to the whims of veteran members at the expense of others. They were going to end up with a tiny group of super happy members. Not ideal.

It’s usually a good idea to compare the Average Member Impact score by the following segments

  • Member location/language.
  • Member tenure.
  • Member type.
  • Member participation.
  • Member satisfaction.

You will usually notice segments of members which have lower or higher CDI. This is where you can focus your efforts to achieve the biggest impact. This is how you build a data-driven strategy.

In this example, you can see there are some groups which are doing well and should be grown, others which need to be improved, and some which may not be suitable for the community.

segmented performance graph

If you use a pop-up poll, you can also see if the AMI score changes by where the poll pops up within the community. You might even find the AMI score varies by the category of discussion people are viewing.

This suggests you can focus on specific features or categories to increase the CDI score.

 

What Impacts Your CDI Score?

Once you have a simple score to measure community success, you can start to measure everything you do against that score.

For example, if you’re changing the community platform, adding a new gamification program, hosting more events, introducing a new onboarding program, hiring more staff, or increasing the budget for the community, you can measure the success of all these things by looking at the Community-Driven Impact score.

Statistical buffs amongst you can also look run a multiple regression analysis to see which specific things most influence the community-driven impact score. This might include:

  • No. posts.
  • Average time to first response.
  • No. accepted solutions.
  • No. posts replied to by superuser
  • No. superusers
  • No. events
  • etc…

You can start to see what the statistically significant predictors of the CDI score are and use these as the primary targets to improve in your community strategy. Combine this with qualitative insights from member interviews and you should be able to build a clear picture of what you need to change in the community and why.

You might also discover much of what you’re doing doesn’t have any impact at all and you can halt those activities. You might also notice that other things are closely correlated and develop programs to change these metrics.

For example, if the number of superusers is strongly correlated with the average time to first response and the average time to first response is correlated with the CDI – you might place greater emphasis on recruiting superusers at the expense of other activities which don’t impact CDI.

 

Community-Driven Impact Sits At The Heartbeat Of A Data-Driven Community Strategy

data driven community

This is how you put together a genuinely data-driven community strategy.

There’s no guesswork or erroneous assumptions. You identify the target, examine what impacts that target, and develop the right programs to influence those factors.

There will also be some trial and error. All data is based upon some assumptions which are prone to error and bias. But it’s a process of trial and error which will always eventually land upon the right outcome.

By having a single metric to guide your efforts, you can identify what the results of the community should be and align all your actions to achieve that result.

If you want to learn more or have us set this up for you, drop us a line.

 

Resources

The Beginner’s Guide To Community Management

Today we’re launching The Beginner’s Guide To Community Management.

This is a project which has been some time in the making. Our goal has been to put together a resource which helps newcomers in the industry quickly get up to speed on the basics.

The guide covers:

  • What community management is (and isn’t)
  • What makes the community unique from other company activities
  • Which tools and software to use for a community.
  • What tasks should be undertaken daily, weekly, and monthly.
  • How best to engage members in a community.
  • What metrics matter (and which don’t).

We’ve also thrown in plenty of resources, frameworks, and examples to help us in the journey.

https://www.feverbee.com/guides/the-beginners-guide-to-community-management/

Whether you’re new to the industry or a veteran looking to brush up on the basics, this guide might help.

P.s. please also share it with anyone who might find it useful

This Is How You Identify Your Community’s Biggest Design Issues

The more research we do, the more I’m convinced most community professionals completely misunderstand how members use their sites and what they feel about them.

I say this having sat through hundreds of sessions where members show us how they use the community.

This misunderstanding leads to huge amounts of time being invested in improving aspects of the community experience that members really don’t care much about. Too much time, money, and resources are wasted which could be better invested in higher impact areas.

I want to share our approach to this problem and what we’ve learned from it.

 

Why The Current Approach To User Experience Is Inadequate

The current approach to developing a better community experience is woefully inadequate because of poor research.

Typically, a community person will speak to a few members, ask them what they like and don’t like, and then try to make improvements based on that. Often this leads to really random suggestions floating to the top of the priorities list.

I’ve seen more than a few organisations spending months revamping gamification programs, member profiles, and adding more groups because it came up in member interviews. Later they will wonder why this had no impact.

The problem is members are bad at describing what they want and need.

There is a much, much, better way to improve your community. But it also begins with having the right data to begin with.

 

Watch What Members Do, Not What They Say

Members are terrible at recalling a community experience and how they felt in the moment. Yet this is exactly the kind of information you need.

The secret then is to be there with them in the moment!

The way FeverBee approaches audience research these days is to sit with members and watch how they progress through a community.

I mean this in almost the most literal sense possible. You have to have members sharing their screens as they progress through the community – and ask them questions as they do.

There are several steps to this process.

  • Step 1) Build a list of members to speak to
  • Step 2) Send out the invites and schedule calls.
  • Step 3) Give them tasks to do and see how they do it.
  • Step 4) Identify and prioritise the issues.

Getting each of these steps right is important to understand how to get this process right.

Resource: Rocket Surgery Made Easy (read this – it outlines the process in detail!)

 

Step One – Building A List Of Members To Speak To

The common mistake is to interview the people you know best. These people are usually the most highly engaged members of the community.

The problem is they experience the community in a completely different way from your regular members.

You don’t want to just talk to your superusers, you want to talk to the people who aren’t superusers (yet). You want to build a list of members you can speak to which includes lurkers, newcomers, irregulars, and veterans.

If your community caters to multiple groups, you want that reflected in your research too. This is especially important if you’re dealing with developers, users, administrators etc…or those from diverse geographic locations.

A big word of warning here, don’t recruit people from places like userinterview.com.

While having a group of random people review a site can give you some interesting insights, they won’t behave the way your members do. They won’t begin with the goals and understanding of the topic.

Resource: Build your list of inteviewees

 

Step Two – The Outreach Email (a simple script)

A clumsy outreach email can undermine a good process.

The best outreach messages run along the lines of

“Hi [name],

I’m reaching out because we’re looking to improve our community efforts in the coming year and I’m hoping you might be willing to help.

We’re looking to get your experience on what is and isn’t working for you – and see how you experience the site. 

Would you be willing to spend 30 minutes speaking with [person]? 

No worries if can’t, but it would really help us if you can”

The goal is to keep it personal and low-pressure. We also try not to over-automate the process. We could easily add a calendly link in the first email, but then it would seem less like instruction and not seeking consent.

If we receive a positive response, we can then follow up with….

“That’s great, thank you so much! 

To save time going back and forth, would you mind booking any slot that works for you using this link:

[Calendlylink here]

During the call, we would love to sit and follow how you explore the community in real-time and get your thoughts as you go. You don’t need to prep anything.

Let me know if you have any questions.”

This lets people select a time slot that works for them. Make sure this is connected to a Zoom link too so people can easily participate.

This process has to be friendly, personalised, and solicit their consent before making requests of members to do something.

If you don’t get a response to initial messages, you can add a $20 gift voucher and see if that works. But we’ve usually found we don’t need to do that. Most people are interested in sharing their thoughts and feeling appreciated.

Resource: Calendly (for easy scheduling of interviews)

 

Step Three – Tasks For Members To Participate In

The next critical set is to give the right instructions for members. These calls are typically 30 minutes long (any longer and we find the acceptance rate drops sharply!).

During this process, it’s important to give members specific tasks to do which reflect what you know most members come to the community to do.

Sure, having members casually browse the site and highlight what they like or don’t like is interesting. You might pick up a few pointers. But this isn’t how most people behave. Most people visit the site to do something specific. There’s a certain mindset which is associated with that.

Here are some of the best examples:

  • “You’re looking to solve [common problem], how would you go about doing it? Why did you go there? Why not the community?
  • “You want advice on the best equipment to use to [achieve goal], how would you do it?”
  • “You landed on a discussion page and didn’t find the answer, where would you go next?”
  • “You want to ask a question, how would you do it?”
  • “You want to find information about [common issue], where would you go?”
  • “You’ve noticed an issue you want us to fix, how would you let us know about it?”
  • “There’s something you want changed about our products, how would you tell us about it”

Now watch what members do as they go about solving these problems.

You will quickly get a sense of the issues and how people go about their choices.

 

Task How frequent is this need? What did they do? Why did they do it? Any issues / notes?
Solve a product problem Monthly I search on Google It’s easiest place to get the best information Perceived community as too much effort to find the answer.
Find peers in my position Weekly LinkedIn It’s easier to search by job title. Haven’t seen this as an option
Find latest expertise Monthly Went to the company page and then latest product. Didn’t see any obvious place to get the information in the community. Release notes is hidden and the language is unclear.

 

You might also notice many of these problems might not begin in the community itself, they might begin elsewhere. People might search the documentation or browse the web for answers before visiting the community. These are also good insights to have and to understand their entire journey.

Resource: Interview Script

 

 

Don’t Reveal The Answer In The Question

Notice in the tasks above, we don’t use the common terminology of a community site. We don’t tell people to join groups, connect with others, ask questions, make feature suggestions etc.

If you do this, members simply scan the text and select the words you just gave them. This isn’t how people use the site. The goal isn’t to find out if members can find the community features you’ve named. The goal is to see if members can intuitively achieve their goals using the features the community presents to them.

You want to know if the terminology is right and if the features are positioned where members can see them, and which features members choose to use for which goals.

This is why we don’t tell people to log in to ask a question, we want to see if they figure it out.

This is also why we don’t use ‘feature suggestion’ and instead use “tell us about it”. They have to figure out which sections of the community best guide us to the answer.

 

Don’t Give People ‘Community-Focused’ Tasks

With some exceptions, most people don’t come to the community to do tasks like updating their profile, viewing the gamification score, or joining newcomer groups.

Asking people to do these tasks gives you pretty useless data. The tasks you give people to do should reflect what members want to do before they arrive at the community.

People often want answers to questions, to find people who can help them, and to find the information they need. You want to keep this relatively broad and see how people solve the issue.

 

Prompt People To Share Their Thinking

As you go through the process, you want to prompt members to share what they’re thinking at any given time.

You don’t need to do this every five seconds, but as you watch people try to solve a task they’re given, it’s a good time just to ask them to share what they’re thinking as they do it.

You might get answers like:

“Well I think I need to register, but I can only see a login button and not a register button.” 

This kind of information was useful because the company assumed everyone would realise that to register you simply click the login button. A simple change of text resolved this.

“I can’t be bothered to browse through so many navigation options, so I’m just going to search and see what comes up”

This suggests that the site is overwhelming and people will increasingly rely upon search to resolve their issues.

Another answer might be:

“This is the 4th page of the registration process, it’s really starting to get annoying. I just want to get going”

This was interesting for a client where the registration process was long and they were trying to overly prepare the member for the experience instead of just letting a member dive in and do what they wanted to do.

You don’t need to overdo this. I.e. asking people to do this every 10 seconds just gets annoying (and starts to seem a little insecure). But it’s a good idea to do this when you notice a pause or a moment of uncertainty.

It’s a good idea to use Otter AI or another transcription tool to automatically convert the discussions to text so you can others can analyse it with the recording later.

Resource: Otter AI

 

Step Four – Identify and Prioritise The Issues

As you begin seeing how members really use your site to solve issues, you start to notice the issues.

For example, you might notice no one is reading your onboarding messages and some areas of the community receive almost no clicks.

You might notice people struggle with the search function and often don’t find the answer they want.

In one client, we noticed that rather than browse the community, they would return to Google and search again for a related term with the community brand name and see the results.

You don’t need to do this alone. If you’ve recorded the sessions as you should be able to watch this with colleagues and get a real feel for how people use and experience your community.

List all of these by how critical the issue is.

Here’s a simple scale you can use.

  • 5 – Showstopper – immediately stops the user from progressing any further.
  • 4 – Critical – has a major negative impact on the member experience.
  • 3 – Annoyance – causes a strong negative reaction.
  • 2 – Frustration – causes mild frustration.
  • 1 – Drag – slows the user from achieving their goals.
Issue Cause Severity
The forgot password feature isn’t working. Uncertain why these emails aren’t going through 5
Community is overwhelming – people don’t know where to ask their questions. Too much information on the homepage. Unclear where members should go next. 4
Doesn’t realise the login option is also the registration option. The term ‘register’ isn’t used anywhere 4
Members can’t find the latest information about products. ‘Release notes’ isn’t a term members are familiar with. 3
Registration process takes too much time to progress through. Too many pages have slowed this process down. 3
Members can’t find the posts they’ve recently made. There’s no natural path outside of member profiles to find posts a member has made. 2
Members are unable to connect with people like themselves. There’s no easy way for members to search a member directory by attributes 1

This is just an example, but you get the idea. The majority of issues here are now ranked and can be resolved.

 

Step Five – Prioritise By Effort and Impact

Now we have a prioritised list of issues, we can start thinking about the solutions to each of these issues.

At this stage, you may need to ask around, spend time talking to developers or your platform vendor, and see how other communities have addressed the issue (or get consultancy support).

For each of the issues you identify, you want to know the solution and how much effort the solution will require. You can develop any scale you like, but we prefer a scale such as:

Time Money Misc
1 – Maximum effort 6+ months $50k+ Procurement required or info sec review needed.
2 – High Effort 3 to 6 months $10k to $50k Vendor support required.
3 – Significant Effort 1 to 3 months $1k to $10k Developer needed.
4 – Low Effort 1 to 30 days $100 to $1k Admin support needed
5 – Minimal Effort Less than a day $0 Can personally implement

 

Notice we have inversed the scale here (5 was highest for severity but lowest for effort). This is because we want to prioritise the tasks which give the biggest impact for the least effort.

Now we can put the solutions and prioritised list together.

 

Issue Solution Severity Effort Priority score
Doesn’t realise the login option is also the registration option. Easy tweak to login/register. 4 5 9
The forgot password feature isn’t working. Needs a developer to review the process and resolve it. 5 3 8
Community is overwhelming – people don’t know where to ask their questions. Need a revamp of the community homepage with separate instances for members depending upon profile data. 4 2 6
Members can’t find the posts they’ve recently made. Need to show members the recent posts they’ve made on the homepage when they visit and in the member profile. 2 4 6
Members can’t find the latest information about products. Need a separate signposted area on the homepage and on discussion pages. 3 3 6
The registration process takes too much time to progress through. Need to reduce redundant pages and reduce the information we ask from members. 3 2 5
Members are unable to connect with people like themselves. Need to create a member directory and solicit metadata from members. 1 2 3

 

Now we have a prioritised list of issues identified, potential solutions, and the level of effort required to resolve them. This is something you can use when you build your community roadmap.

You can add and change this over time, but it’s always a good idea to do it.

 

Counter-Intuitive Discoveries You Should Know

In the time we’ve been doing this for clients, we’ve made some discoveries which seem to apply to most communities. So here’s a cheat sheet of things you might want to think about when developing your own community experiences.

  1. The majority of visitors read almost nothing. The overwhelming majority of visitors won’t read more than 2 to 3 words on your homepage. They scan to find the ‘thing’ of interest and then jump to that. You should reduce static homepage text to a minimum.
  2. Members really struggle to follow conversations. Members really want to find the discussions they’ve participated in previously (to check for updates), but they often struggle to do so. They forget what category they posted in and can’t find it from the homepage. This is a huge frustration. It needs to be dead simple for members to find a list of discussions they’ve participated in.
  3. Members switch to a different site quickly. Rather than spending just 5 to 10 seconds trying to get your SSO or two-factor authentication to work, they’re more likely to switch to a different site to find what they need. It’s just too much effort. Sites which have a tendency to log people out often for security reasons cough Salesforce cough struggle with this most.
  4. Most people make a couple of searches for the answer. Asking a question is usually a last resort. Most people with a question will make a few searches and then publish a question.
  5. Members post questions in multiple places. People care more about getting an answer than where they get an answer. Around 50% of people seem to post questions in the community and support at the same time.
  6. Average visitors browse the homepage for 5 to 10 seconds. The average person wants to feel a sense of progress. They browse a homepage for 5 to 10 seconds and then click on something (anything), to feel a sense of momentum. People hate feeling stuck.
  7. Only a tiny percentage of regulars scan the latest discussions. This really surprised us. Despite the common practice of putting the latest questions at the top of the homepage, very few people scan them. Most consider them irrelevant to their problems right now.
  8. Members browse top-level navigation before searching. Most members view what’s in the top-level navigation and click the word that most closely resembles what they’re looking for. What appears in the navigation and how the navigation is structured is critical. Members must be easily able to browse for what they need.
  9. Almost nobody seems to look at the leaderboards. Even the people on the leaderboards don’t seem to look at them often. This is especially the case for static leaderboards based upon cumulative activity since launch as opposed to monthly leaderboards or the past 30 days.
  10. No one reads the welcome message. It really doesn’t matter what appears in the welcome message unless people are confused about the nature of the site. If they don’t understand the community itself, they might glance at the welcome message to comprehend what it’s about. But these are very rare situations.
  11. No one reads the code of conduct. Theoretically, you could put terms and conditions you like in there and people would still agree to it. Although, interestingly, they will read articles titled “The Five Rules Of [community]”. So, maybe give it more of a Fight Club vibe?
  12. Popular discussions. Members generally like seeing the list of popular discussions in a community. This list works best on individual discussion pages as much as on the homepage. However, once a member has seen the list once, they dislike seeing the same discussions again.
  13. Related discussions matter a lot – but only in high-volume communities. At low volume, the related discussions usually aren’t relatable enough. They simply appear because something has to appear.
  14. Recent activity really, really, matters. If the community seems dead, few people ask questions. Recent activity labels are far more important than any static copy/text on the page. No one can recall the liveliness statistics, but they might build up into a more subconscious understanding of how busy the community is.

Even these findings – most of which surprised us – are really only the tip of the iceberg.

The more research you do (with an open mind) the more interesting discoveries you will make.

As long as you’re willing to sacrifice some sacred cows, you can rapidly improve your community experience by following the member journey.

 

Get FeverBee To Undertake Your UX Research

If you don’t have the time or experience doing UX research, we can certainly help. It’s a service we’ve done for many clients and we’ve been able to show rapid improvements.

In this capacity, we can do deep benchmarking of your community experience, undertake interviews with a range of members, and build a detailed list of improvements.

If you like, we can also take this a step further and undertake the design and development work ourselves to bring the community up to the highest possible standard.

If you want help, you can click here to contact us.

 

Resources

Brand Communities Change Customer Attitudes – Here’s How You Prove It

It’s possible to prove the value of community by showing how behaviour is changed.

For example, you can show the impact of a community upon retention, call reduction, or some other behavioural metric.

The problem is it’s really damned hard and you’re never going to have a satisfactory result.

The simplest way to show the value of a community isn’t to prove people behave differently – but people think differently.

It’s a lot easier to measure meaningful changes in member attitudes than meaningful changes in member behaviour.

 

Where Does The Real Value Of Community Show Up?

Another way of thinking about this is to consider where the real value of community shows up.

Take the incredible volume of customers you support. Most people measure this using ‘call deflection’. But a growing body of data shows that people are asking questions in several channels at once and they often ask questions in a community they wouldn’t bother calling support for.

It’s not right to measure this in deflection dollars. But where it will show up is in how people feel about using the products. If they get better results, feel more satisfied, or get unstuck quicker – that shows up in their attitudes.

And this is the most direct benefit of a community – it changes member attitudes.

It’s often quite tragic changes in attitudes aren’t measured. Attitudes drive behaviour.

If you can change the attitude, you can change the behaviour.

 

What Attitudes Can We Measure?

Let’s imagine you went to your boss tomorrow and presented real, live, data that showed, that since joining your community, members have significantly improved satisfaction with products and the likelihood to recommend you to others.

Imagine you could show your community really moved the needle in how people feel about you and your brand and future purchase intent? That’s a powerful metric. Imagine the associations they have with your brand have changed in a more positive way?

Or, for the golden ticket, imagine you could show that brand preference has markedly increased vs. any competitors. Now members consider your brand superior on a range of different attributes.

Or, if you’re working for a non-profit, imagine you could definitively prove that a member’s quality of life scores have increased. Perhaps you could also show they now feel they are better prepared to handle whatever circumstance they find themselves in since joining your community?

In more specific terminology, we often use terms like:

  • Member Satisfaction. (CSAT)
  • Net Promoter Score (NPS)
  • Quality of life (non-profits)
  • Brand attitude
  • Brand perception
  • Brand preference

But it all rolls up into the same key measure – attitudes have changed.

This is the kind of data that is a lot easier to get and far harder to ignore.

 

How Do We Measure A Change In Attitudes?

You probably know the common problem with any metric you want to change by now.

You can easily show any community members score higher in almost any metric than non-members. It’s a lot harder to show it’s the community that caused the metric to be higher.

For example, how do you not know it was simply your best and most loyal customers who joined the community in the first place? Establishing any kind of causal relationship which removes the possible presence of confounding variables is difficult.

But these are some relatively simple solutions. They’re not expensive, but they do take time and you need to be careful with setting up the process correctly.

 

Option 1 – The Controlled Trial

The best solution is to do a controlled test. This is hard to do but it’s not impossible. In this approach, you would segment non-participants at random into two groups. You can send an email to one group with an invitation to join the community and track the results against the other group.

Or, better, only enable one group to see/join the community and not the other group.

In practice, however, this is problematic. For starters, it’s damned hard to stop people from seeing the community. You would have to remove it from search results and configure the technology for the community not to appear for a large group of members. Few organizations are keen on that. And even then you might only be measuring the differences in the characteristics of membership of each group.

If you can do a controlled test, great. But if you can’t, I’d suggest a simpler method…

 

Option 2: Gather Data When Members Join

The reason why few can measure the results of the community is they only collect data based on their timeline instead of the members’ timeline.

It’s a little like – ‘hey it’s October! Time to run our survey!

But there’s nothing particularly special about October (or any month of the year) which is especially good to run a survey. In fact, you’re very likely to bias the outcome by limiting your survey to once a month.

This is why it’s far better to measure member behaviour by their tenure. Specifically, a member hits a certain milestone, it’s a good idea to hit them up with a quick survey to measure their attitudes.

The most important of these milestones is when members join! If you get a baseline attitude survey when members join, you can compare it with later surveys and estimate the results. If you ask the same questions again a year later, you get to see the impact of the community.

 

Why It’s Critical To Capture Member Data When They Join

Let me share an example of why capturing member attitudes when they join is so important.

Last year, I worked with a client whose community reported the lowest NPS of any customer support channel. We’re talking deep in the toilet low!

The community team was getting a lot of criticism for the poor performance of the community.

But was it the community’s fault? 

We began doing some exploring. It soon became clear the community was simply attracting the most frustrated members. These were members who had had negative experiences in other channels and were turning to the community as a last resort.

They had the lowest attitude scores before they even joined the community!

The community was still helping these customers. But they often had problems for which there wasn’t a solution. The community helped them realize that. They weren’t happy about it – but at least they could keep looking.

The problem was no one had taken the time to capture members’ attitudes upon joining. They had no idea if the community had improved the results or not.

(aside – and remember here people who have a negative experience tell several times as many as people. Preventing negative word of mouth is a huge benefit).

Now imagine if they had captured the NPS, CSAT, or other scores when members had joined and could compare progress 6 to 12 months later. You can start to get a sense of the impact of the community.

 

Communities Naturally Attract Members Who Like You More

For example, in the graph below, you can see how much the NPS scores vary by each category of members. For example, avg. non-members today, avg. non-members last year, average new members a year ago, avg. new members today. As you can see here.

graph showing how Communities Naturally Attract Members Who Like You More

This is interesting data, but it doesn’t really show the impact of the community. It more likely shows that people with higher NPS scores might be more likely to join a community and those who like the brand most stay in the community for longer.

What we need to know is the difference between non-members and first-year members over the same time frame. This is where the data starts to get interesting, as you can see below.

graph showing difference between non-members and first year members

The NPS varies (remarkably wildly) from one month to the next. But over the course of the year it appears that community members appear to have a higher difference over that same time frame. You can see this here:

graph showing impacts of Community driven change

Sure there is plenty of variability, but there’s a clear trendline here. Community drives a higher NPS score.

It’s worth noting this isn’t 100% conclusive.

You might simply be measuring the people within each cohort who bothered to complete a survey a year later – people who might naturally be more predisposed to completing a survey. However, by using random survey sampling and offering a small incentive, you should be able to overcome much of that.

This is often known as the ‘difference in differences’ approach. You compare groups and track the impact over time as you see below.

Quick caveat here, if you’re working with a tiny community, this might not work for you. You probably can’t get enough responses without offering some substantial incentives. But for any community which has more than a few thousand members, you should be able to undertake a rotating study and show the results.

 

How Do We Measure Results?

Before we measure the results, let’s understand what we’re measuring here. Metrics like Net Promoter Score (NPS) and Customer Satisfaction (CSAT) have been extensively covered before.

Let’s focus on the three lesser-known attitudinal metrics we want to cover. Ones which are real game-changers for organizations.

Brand Perception

I think of Apple as stylish, easy to use, and expensive.

I think of Logitech as basic, functional, and simple.

I think of Google as useful, friendly, and accessible.

You probably have (very) different perceptions of all three.

But we aggregate the results from a large enough group of people and you will soon start to build a great understanding of how people perceive of each brand.

These perceptions matter a lot. They are a leading indicator of future purchase intent, retention, and likelihood to promote the community.

So imagine your community could profoundly change how people perceive your brand.

Imagine if when members joined the community they used terms like confusing, overwhelming, and fiddly. Then a year later they used terms like automated, supportive, and comprehensive.

In less than a year, you can show you have completely changed members’ attitudes about your brand.

This is incredibly valuable data which too few people ever measure.

 

Brand Preference

Perhaps the only thing more valuable than changing perceptions is changing preferences.

Brand preference is where you compare your brand against those of a selected group of direct and indirect competitors and get feedback from your members on how they rate each of you. It’s one of the most common and powerful impacts of community – but few people ever try to measure it!

Imagine you can go to your boss tomorrow and say because of your community, members have now increased preference of your brand against competitors by xx% points. Or perhaps they simply associate your brand with more positive terms than any competitor.

If you can show that 10,000 people have improved their preference for your brand since being in the community, it becomes a no-brainer to try and get as many people engaged in the community as possible.

Aside, the other major benefit of this kind of research is you get a unique insight into how people think and feel about your brand. I’m often amazed that organizations don’t constantly do these kinds of surveys given just how remarkable the results tend to be. You can provide your marketing and PR teams with precisely the terms they need to use and messaging to deploy to achieve the results they want.

Second aside, you can also take this a step further and measure the relationship between brand preference and purchases in the future to see exactly how strong this connection is.

 

The Danger of Measuring Brand Attitudes

Let’s quickly highlight an obvious danger of measuring member attitudes.

The obvious danger is the results might show you member attitudes declined since joining the community. On the surface, this suggests the community is having a negative impact.

This is a risk, but even if it is true, you 100% want to know this!

Once you have this data, you can begin interviewing members and drilling a bit deeper into your data to determine the precise cause.

For example, you might find it’s a specific segment of members having a negative experience.

In one client, when we drilled deeper into the data, we found it was people having issues with one particular product line. They were never likely to the solution they wanted because there wasn’t a feasible solution to the major problems they were raising. They felt they were wasting their time on the community and we were seeing hundreds of questions go unanswered.

We came up with a simple solution. If the community couldn’t help with these kinds of issues, it shouldn’t support these kinds of issues. We directed these questions directly to other support channels and closed down this section. Any time you notice the community is doing more harm than good, you can figure out why and take action.

 

What About Non-Profit Communities?

This might all sound good if you work for a typical business, but what if you’re working for a non-profit? Does any of this apply to you?

I’d argue it applies more to non-profits than any other organization. The only difference is you’re measuring different attitudes. Instead of looking at brand attitudes, you’re looking at more representative metrics such as quality of life, increased capacity, or overall confidence in managing their circumstances.

A Quality of Life Survey, for example, can show the impact of community across a range of well-being factors such as health, relationships, finances, and overall life satisfaction. Sometimes the impact of a community might appear in places you don’t expect.

 

Setting Up The Survey (Or Poll)

You can set up a survey to capture data both before and after someone has joined a community.

Ideally, you want to have the same people participating in surveys each year. A 1-year frame can work.

One way of doing this is offering a small incentive if they participate in the survey when they join and then offering them a larger one if they complete the survey a year later. Another option is simply to compare the average of newcomers vs. veterans and assume nothing else changed during this time period.

 

Questions For All Surveys

You can adapt these to your situation and environment. You may want to ask additional identifier questions about the type of customer or individual so you can drill deeper into the data later. You might ask questions about the type of products purchased, gender, age, location etc…

Typically the key questions should include:

Q. For approximately how long have you been visiting the community?

(This helps you to separate members who have just joined from those who have been around for a while. If you’re triggering the survey by member tenure in the community you can skip this question)

Q. How would you rate your overall satisfaction with [the product/community]?

  • Very unsatisfied // Unsatisfied // Neutral // Satisfied // Very satisfied

(This is a question which can help establish overall satisfaction with the community environment. You can skip it if you’ve got too many questions, but it can help establish if the community is driving an improvement in results)

Q. If you would like to receive [discount/benefit], please enter your email address below

(This is simply a tracking question to be included at the end. It helps you identify who is participating in the survey and match it up to your customer database. Sometimes you don’t need this, but it’s usually useful to be able to follow up with people who took the survey the previous year)

 

Example Questions For Brand-Attitude Surveys

If you’re running a survey on brand attitudes, you might ask a combination of the following questions.

(NPS qs.) Q. On a scale of 0 to 10, how likely are you to recommend [brand] to a friend/colleague?

This is the standard NPS question. 

(brand perception qs) Q. On a scale of 1 to 10, Please rate how well each of the following traits describes our brand.

List of traits here. This might include a mix of attitude and behavioural traits. For example:

  • Value for money.
  • Broadest feature set.
  • Better quality than other brands.
  • Innovative.
  • Caters to my unique needs.
  • Easy to set up and use.
  • Great customer support.
  • Great documentation.
  • The staff seems friendly and supportive.

You can add any set of attributes you like here which people can use to evaluate your brand. Try to be as specific as possible. 

Aside – An alternative approach begins with the attributes you want to learn more about and asks members to select which brand most relates to which attribute. This is useful if you know specific attributes are most important.

Resource: Brand Perception Questions

(brand preference qs) Q. Before purchasing from [brand], which other organizations did you consider?

This is good for knowing who you should be comparing against and you can then program each of these options to appear in the evaluative set to ensure you’re not asking people to compare brands they never considered. 

(brand preference qs) Q. How important or unimportant were the following factors in your decision to purchase from [brand] rather than any other brand?

  • Extremely important
  • Very important
  • Moderately important
  • Slightly important
  • Not at all important
  • No opinion

This question will help you identify the key factors which drove the purchase decision. List the most likely factors here and include a rating scale along the lines of

(brand preference) Q. What (if anything) might make you switch to a competitor? 

You can also suggest possible answers and add an open text box. But it’s generally better to let members complete the answers themselves. 

(brand preference) Q. Which brands would you most associate with the following attributes

  • Value for money.
  • Strong customer support.
  • Comprehensive feature set.
  • Easy to use
  • Etc…

This is the critical question. It’s a direct comparison question where you use the relevant brands provided in the previous answer to populate answers for people to complete. Zero in here on the attributes which you feel best drive purchasing behaviour – you can use the answers from above. 

Resource: Brand Preference Questions

 

Example Questions For Non-Profits

Developing questions for non-profits is a little trickier. Every non-profit supports a different circumstance and ideally, questions should best address that circumstance. This may include:

[non-profits] Q. How would you rate your ‘confidence to handle [circumstance]?

  • Not confident at all
  • Slightly confident
  • Moderately confident
  • Somewhat confident
  • Very confident

This is the simplest question to use. It provides a simple snapshot answer where people can rate their level of confidence on a single scale and you can track results over time. You should be able to see precisely the impact of the community. 

[non-profits] Q. How would you rate [quality of life, mental health, physical health, social life etc..] over the past 4 weeks?

  • Poor
  • Fair
  • Good
  • Very good
  • Excellent

You can repeat this question with several variations above to capture the full impact of the community across a range of factors. This lets you identify where the community has shown results in areas where you might otherwise not expect. You can go further to deploy a full quality of life survey using the resource below. 

Resource: Quality of Life Questions

Resource: Writing Survey Questions

 

Next Steps

Behaviour is primarily driven by attitudes. Your community shapes and influences those attitudes in a major way. You’ve probably had countless interactions which you know have had a major impact on your audience – and you probably haven’t tracked the outcome of those interactions.

Notice we use the word ‘track’ rather than ‘measure’ here. Measuring gives you a snapshot of today. That’s interesting, but not very helpful. Tracking attitudes over time will help you understand and prove the impact of your community.

Imagine how powerful it is to have results like this to share with colleagues:

However, if you want this to work you have to set up the data properly. You have to prepare the dataset today for the results you want to show tomorrow (or 6 to 12 months from now).

When execs are against the community, they often dismiss the ROI data which can be complicated and prone to all sorts of attribution issues. However, it’s harder to dismiss attitude data – especially when it’s been properly collected and analysed

Here are some simple next steps

  1. Decide which attitudes you’re going to track.
  2. Setup the survey questions and test the survey on a small audience.
  3. Setup the survey to trigger members based upon tenure (time in or no. visits to the community).
  4. Automatically pull the data into a visualisation tool (Tableau, Looker, PowerBI etc..)
  5. Drill into the results to determine which segments/factors most impact the results.

 

Let FeverBee Calculate Your Attitude Change Score

All of the above takes a lot of effort to set up and ensure the data is collected properly. This is where FeverBee can take care of the process for you. We’ve worked with plenty of clients to measure and analyse the impact of their communities.

We can take on the entire process and simply provide you with the data you need to prove the value of your community.

If you want help, drop us a line.

 

Key Takeaways

  1. It’s easier to measure meaningful changes in attitudes than behavior.
  2. Track attitudes over time – especially when people join.
  3. Brand perception, brand preference, and quality of life are valuable data points to have.

 

Resources

 

What Is A Login, Post, Or Download Worth In A Community? (An Analysis by FeverBee)

It’s incredibly hard to prove the value of a community.

Unless you have the time, expertise, willpower, and capabilities to do a controlled trial; most data analyses establish a relationship. The direction of that relationship (and whether there are confounding variables) is often left unexplored.

However, this doesn’t mean you can’t gather some really interesting insights. You may not be able to definitively prove the value of the community, but you can estimate the possible value of the community using statistically valid methods.

 

Pragmatic Institute Needed Help To Measure Their Community

In March 2022, The Pragmatic Institute hired us to help them determine the ROI of their community.

The Pragmatic Institute is a provider of expert training for Data, Design, and Product Teams. The Pragmatic Institute primarily sells to organizations – that often subscribe multiple staff members to courses at any given time.

To support their work, they created an online community hosted by Higher Logic. The community was popular and active. Members were clearly getting value from participating in the community. However, it had proven difficult to show whether engaging in the community led to more sales or better outcomes for The Pragmatic Institute.

FeverBee was tasked with answering three key questions:

1) What is the connection between community participation and the number of courses taken?

2) How might community activity impact training revenue?

3) What kind of community behavior should be encouraged to improve the metrics above?

 

Methodology

feverbee community methodology

Step One: Preparing the dataset

The first step of the process was to build a proper database.

We began by preparing the three datasets:

1) The dataset of Pragmatic users from Salesforce.
2) The community activity dataset from Higher Logic
3) The sales data from Salesforce.

Like many organizations, member and customer data was scattered across multiple channels.

The community was hosted on Higher Logic which contained all the engagement-level data. Purchase data lived in one set of Salesforce reports while course completion data lived in another.

To make things even more challenging, the individuals who took the courses weren’t usually the people who paid for them. Finally, the data often contained inconsistencies in how the names of organizations were recorded. Sometimes acronyms were used, other times the entire name was used.

Over several weeks, we cleaned the data, identified and resolved missing variables, and analyzed the outcomes.

 

Step Two: Running comparisons

Next, we began running comparisons between different segments of members. This included comparing active members with inactive members. We also compared members who have engaged in the community with those that didn’t.

When variables had a sufficient distribution, we performed linear regressions to look for a linear relationship between behavior and revenue generated per organization. We also grouped members by the number of posts they had made to determine if a specific number of engagements was associated with an increased probability of future sales (this is known as ordinal regression)

 

Step Three: Presenting Results

Once we had undertaken this analysis, we examined and visualized the results to estimate the potential value of the community. This is what we found…

 

What Is The Pragmatic Community Worth?

How Much Is A Community Worth?

Overall, almost every behavior was associated with greater revenue.

  • Active members (>1 post) generated 21% more revenue than inactive members.
  • Members who make at least one post contribute 44% more revenue.
  • Members who make at least one thread contribute 35% more revenue.
  • Members who log in to the community at least once contribute 21% more revenue.
  • Members who have replied to at least 1 direct message contribute 38% more revenue.
  • Members who contribute a document contribute 38% more revenue.
  • Members who download a document contribute 20% more revenue.
  • The more frequently members log in, the greater the revenue they generate.

By our calculations, community participation was associated with over $1.5m in additional revenue.

 

Which Behaviors Were Most Valuable?

It’s one thing to know the value of the community, it’s another to dive a little deeper to determine which behaviors are more important in driving that behavior.

In our case, we wanted to know which community behaviors increased the probability of members taking an additional course.

Here the data showed three important metrics.

Community - which behaviors were most valuable?

  • Replying to a DM increased the probability of taking an additional course by 16%.
  • Creating a document increased the probability of taking an additional course by 23%
  • Creating a thread increased the probability of taking an additional course by 201%

 

Setting Priorities

The probability of taking a future course was directly related to downloading documents, creating threads, and replying to direct messages.

This helped set some clear and specific strategic priorities within the community.

These priorities included:

  • Encouraging members to create threads
  • Creating more documents to download.
  • Getting more responses from members.

 

The Value Of Community Intelligence

There are far too many community teams who know their community is incredibly valuable but are unable to prove it.

As we’ve seen with the Pragmatic Institute, even if your data is scattered across several channels, we’re still able to combine this data and evaluate the impact of the community upon revenue, attitudes, and almost any other desired outcome.

If you can clearly define the variables, you can usually measure them. Better yet, we are able to specifically show which behaviors are most important and should be encouraged in the community at the expense of others.

As we’ve seen before, this helps you build a solid community strategy.

Once you can prove the value of the community through statistically valid methods, it becomes a lot easier to gain both internal validation and additional support.

If you know your community has value but you’re not able to definitively prove it yet, I’d suggest getting help from our community intelligence services.

Have You Prepared Your Community For These Powerful Trends?

History is littered with organizations (and entire industries) that failed to adapt until it was too late.

The community industry isn’t immune from this fate. Many of us are using community technology and offering experiences that really haven’t changed much over the past decade (or two).

But now major forces are emerging which will permanently change the experience we need to offer our members. If we don’t adapt fast enough, we risk consigning ourselves to irrelevance. It’s time to update our technology roadmap, and do it fast.

 

The Major Forces Shaping The Future of Community Technology

To figure out what should be on your community technology roadmap, we first need to know what the big trends are and why they’re relevant.

Everyone will have their own opinions, but I think these are the ones that really stand out to us.

macros trends in communities

Some of the above are critical trends already impacting our work on a day-to-day level, others are slow-boiling micro-trends that might affect us in the future (and we need to prepare for them now).

 

What Are The Short, Medium, And Long-Term Community Tech Implications?

Every trend above will impact your community technology needs in a different way.

It’s hard to predict the exact impact of each trend, but based upon our best guess, we would advise clients to consider the following updates to their community experience:

community technology roadmapWe’ll explore each of these in detail below:

 

Short-term Implications (within the year)

This section includes the improvements which should be on your roadmap to complete within the year. These are generally inexpensive and don’t require a huge amount of time and resources to implement.

 

Build A Better Community Dataset

Community professionals have been grumbling about the analytics of community vendors for decades. The complaints tend to fall in two areas. Either the vendor doesn’t show the data the client wants to see or the client can’t pull the data from the vendor into another platform. The reality is vendors will never provide the analytical insights many clients want. It’s simply not what they’re set up to do.

But two things are changing now. The first is the rise of community intelligence tools like CommonRoom and Orbit. These have some useful features like creating cohorts based upon member behaviors, being able to instantly get a picture of how the community is doing, and easily export the data into a CSV.

Another major benefit is they help create better datasets which combine social media data with community data. We might argue whether social media ‘is community’, but in countless surveys and member interviews over the years, we often hear responses like:

“Yes, I participate in the community on Twitter, YouTube, Reddit, and sometimes the forums”

Members have already defined it for us. It makes sense to offer a united, connected, experience. A top member in the community shouldn’t be treated like a newcomer on social media channels (and vice-versa). Better yet, having this data lets you build a more comprehensive picture of who your audience are and what they want.

The first step then is to select and utilize one of the two community intelligence tools above. Now you can integrate the accounts. Orbit premium even has a partnership with Clearbit which will let you enrich your dataset from LinkedIn and other channels.

You should be able to have a detailed dataset which includes:

  • Member name
  • Email
  • Location
  • Company
  • Job title
  • Twitter
  • Twitter followers
  • Discord
  • Community vendor username
  • Slack username
  • Language
  • Activities
  • First activity
  • Last activity
  • Date created

This will be the first step to creating a ‘dream’ community dataset we’ll discuss later. For now, you should have this data to hand.

 

Better Engage Newcomers

Community platforms generally don’t do a good job of helping you engage newcomers. You can either send out an automated message to the many (or series of messages) or use the first post notifier present in some platforms to welcome members when they make the first post.

The problem is neither is especially effective. That’s partly because of the nature of the issue. In support communities, most people don’t want to do anything other than get an answer. Bombarding them with poor interruptive messages won’t help. Instead split your audience into two groups.

1) People who joined more than a week ago and didn’t participate.

2) People who made their first post in the community.

This is easy to do in either of the two community intelligence tools.

For the newcomer non-participants, send them a message along the lines of:

“Hi [name],

I noticed you joined our community last week but haven’t been involved much yet.

Is there anything you’re looking to learn? People you want to connect with? Or anything else that would help? Just let me know.

We also have a group for newcomers who want to quickly get up to speed. It might be useful!”

Aside – we’ve noticed when people have a direct interaction with the community manager, they’re far more likely to engage in the community.

For the people who made their first post in the community, send them a message along the lines of:

“Hi [name],

Did you get the answer you wanted from [question]?

If you have a second, it would really be nice if you can let the person who offered a solution know if they solved your problem or not.

This also helps others who read your question know if the solution worked or not (and helps our product developers too know what to fix!)

You can also mark the question as an ‘accepted solution’ by using the tag at the top of the page.

Thanks!

P.s. If you’re just getting started, you’re also now welcome to join our group just for newcomers who want to quickly get up to speed.”

See: Tripling Member Retention

If you’re worried that this isn’t sustainable, engage your volunteers to help you.

 

Identify Current And Future Top Members

There’s a lot of academic literature which predicts who the top members of the future will be. As a general rule, it’s people who make multiple contributions on multiple posts quickly after signing up to the community.

You can set up a filter in CommonRoom for example for members who joined within the previous 28 days and are in the top 1% of active members (or you can set an exact number of topics/replies they need to give).

community filters

Each month you can reach out to members in this segment and see if they want to be more involved in the community and check their level of expertise. You don’t have to boost them straight up to an MVP level, but you can give them smaller tasks to see if they might be a good fit.

This can be a very gentle message to let them know they’ve been seen and recognised. For example:

“Hi [name],

I really loved your post about [topic] last week.

Did you find [clarification question]?

I think our members might really benefit from learning more from your experiences. Would you be interested in being more involved in the community?

I’m thinking it might be great to have you [run a group, write a regular blog, host a meetup, be responsible for a topic]?

Let me know if you’re interested. I’m happy to speak further”.

The goal is to build up a pool of potential MVPs you can bring into the program once they’ve shown they have the expertise and the right personality to engage in the community.

 

Reactivate Members Who Are Drifting Away

This is one of the best uses of community intelligence tools right now.

You can identify which members were previously highly engaged and drop them a personal note. Don’t try to automate this. Top members are too important. Instead, you can use a message like that below to find out a) why they departed and b) to gather their input on a topic

“Hi [name],

Haven’t seen you in the community in a while, I hope you’re doing ok?

If you have time, I’d love to get your expertise on a question/event/issue we have in the community.

@username is trying to [question]. Given your experience in [topic], I thought you might be able to help. No worries if you’re swamped, but any thoughts would be appreciated”

The key points from the message above is it’s personal, it’s undemanding, and tries to make the member feel like an expert for the experience they have within the community.

Both CommonRoom and Orbit will let you find members who were previously in the top 25% of active members and are now drifting away. You can even setup alerts to make this a weekly practice for the community team.

 

Eliminate Most Automated Emails

A quick aside on automated messages here.

Of all the new things you can do with community intelligence tools, automated emails are the most overrated. It’s certainly possible to design complex automation processes to appeal to every segment of your audience and reflect their past contributions.

In practice, automated emails rarely have any significant impact. At best, people ignore the emails. At worse, they begin to ignore the source of the emails altogether. Automated emails are not the silver bullet you might imagine.

You can safely remove nearly all of them and not see any negative impact.

 

Kill Your Customisations

Customisations are a lot like buying a boat.

The two best days are when you get them and when you ditch them.

The dates in between are largely filled with a lot of stress. They’re difficult to maintain and you never quite seem to get it to work the way you want it to. Even if they do, they’re rarely as widely used as you think they will be. Whenever possible, kill the customisations. They’re not worth the bother.

As no-code/low-code options increase, drag and drop options will replace coding and more complex technical developments. You may want to hold back on a big relaunch for a while. If you really need a specific feature (and believe me, you probably don’t), you can find a tool that does the feature really well and send people there. Don’t try and make two things fit together which really weren’t designed for one another.

If you have painful customisations, consider phasing them out in favor of dedicated technologies.

 

Intermediate-Term (within 1 to 3 years)

This section covers some of the improvements which will require additional time, resources, and some difficult decisions. This involves setting the community up for a more challenging internal and external environment – one where security, data, and integrations are increasingly important.

 

Search And Destroy (or Remove)

The user experience of hosted community platforms is going to fall further and further behind that of mainstream social media. Given the choice of where to participate, people will choose the big social media platforms. They’re simply slicker, less concerned with strict security rules, and easier to use.

This means hosted community platforms are increasingly like Wikipedia. They’re clunkier and not as aesthetically pleasing as most web pages – but it’s the king of search traffic. Optimizing for search is going to be critical.

This means improving technical search and ensuring the community itself offers the best possible experience for people searching for information. A major part of this is archiving content no one visits anymore.

I’d suggest beginning with content/discussions which:

1) Attracted less than 10 visits in the past month.
2) Have received less than 2 posts in the past year.
3) Have been published more than 2 years ago.

Aside – the wild thing is most people don’t even consider search when deciding what platform to use. Yet, our experience suggests the platform has a big impact upon the level of search traffic you can expect. Budget $10k to $20k for search optimization.

 

Secure ISO27001 and SOC 2 Compliance

Infosec teams are gaining veto-rights on new and existing community platforms. Ensuring your platform is compliant is becoming an increasingly important issue. As fines for infosec breaches grow, compliance is going to become an increasingly big issue. This makes selecting a platform with high security requirements evermore important.

ISO27001 and SOC 2 are similar but not the same. The former is a set of standards which define an information security management system (ISMS), the latter is evidence that security management systems have been implemented and are functioning well.

Most of the major platforms are working on them. Generally speaking, the bigger the platform, the more likely it is to have the compliance certificates you need.

 

Create and Optimize Your Workflow (for high-volume communities)

You really shouldn’t be managing a high-volume community by the front-end (i.e. using the same interface every other member uses). It’s simply darned inefficient to manually click, reply, and/or delete posts. It also leads to important posts slipping through the cracks (i.e. new activity pushes posts to the top of the page as you scroll down).

It’s also terrible for escalating questions and issues to colleagues. I know too many community professionals who manually email discussion links to colleagues (usually product folks) and wait for an answer. That’s far too much effort.

Beyond a relatively low level, it makes sense to focus on choosing a platform (or developing a system) which enables an easy back-end process for engaging members and responding to discussions. This means having posts shown in a stream which can be filtered by a variety of criteria and allow for easy tagging, assignment, escalation, and approval of various discussions and content.

You can get an extremely hefty performance bump simply by choosing a platform with the right workflow or developing one manually.

 

Integrate with the CRM

Everyone wants to ‘integrate’ their community with other platforms, but it’s not often clear what that makes in practice. So it’s a good idea to get really specific about what that ‘integration’ means. Typically it means something on one platform doing something on the other (and vice-versa).

You might want some combination of the following functionality:

  • The community platform being able to create new contacts on the CRM.
  • The community platform to automatically update the CRM when a member updates their profile data (i.e. photo, name, email, company, location etc…)
  • To have a button that appears to admins alongside community posts that enables the post to be escalated as a support case in the CRM.
  • To have a button that appears to admins alongside member icons to create a ‘lead’ in the CRM for others to follow up on.
  • Add member activity feed data to records on the CRM.
  • Use CRM data to suggest tags in the community platform when members ask a question based upon the products/services they use.
  • Show community data on member records (date joined, date participated, last date participated, total number of posts, member rank etc…).
  • Show CRM data (i.e. courses completed) on member profiles.

This isn’t a comprehensive list, but it gives you an idea of what you might be looking for.

 

Added Language Accessibility

It’s almost nonsensical not to add language translation options to your community.

Members in one country should automatically see posts published in another language automatically translated for them (with the option to disable this if they like). Sure the translation might not be 100% perfect, but it would make the community accessible to people from around the world.

Given the advanced translation capabilities recently, this is a great option. We recently added this to a Discourse community. When combined with members’ locale data, it automatically translates posts into the language based upon where the people are participating from. This massively increases the accessibility of the community to people from around the world.

This alone could significantly increase engagement within the community.

 

Advanced Data Analytics And Automated Reporting

As we discussed here, we’re not even close to getting the level of insights and engagement we should be from our community platforms. Analytics will never be a great feature of platforms (but being able to easily pull the data into another platform should be).

You need to invest a significant percentage of your time and budget into getting good community intelligence. This usually means things like:

  • Automatically pulling community (and CRM) data into a data warehouse like Snowflake to clean and seamlessly use data across a variety of services.
  • Using tools like PowerBI and Tableau to automatically update and showcase your analytics.
  • Publish monthly ROI reports which shows current community value and predicts future metrics based upon previous trends.

You should be able to constantly show the impact of community upon retention, churn, sales, deflection, and in a variety of other areas. You should also be able to model the impact of, say, spending $50k on social ads to drive more people to the community or increasing the number of MVPs by 20 etc…

Even better, have a validated model (like the one below) which highlights how the benefits of a community are achieved and what the moderating variables are.

how do communities build ROI

You can use our community intelligence services to set this all up for you.

 

Long-Term Roadmap (3+ years)

This section covers where the long-term trends are heading and how best to harness them to deliver the best community experience for both members and the organization.

I’m far less certain about these areas than the former two – but that’s the nature of predicting long into the future.

 

AI-Assistance For Handling Problematic Content

For high-volume communities, you should have automated rules for handling problematic content. Some community platforms (e.g Discourse/Khoros etc…) have basic functions included which ban posts from members which are typed too quickly or published too soon after joining.

What we should be planning for is a system that learns how you handle flagged content and makes recommendations for you. The system should increasingly be able to make its own predictions about what content to remove and people to ban with an estimated degree of certainty. Now you might set upper and lower thresholds (say above 80% and below 20%) above which the suggested rule is automatically executed and below which it is not executed while you review those in the middle with a simple option to accept or reject the AI suggestion.

You can then skim through executed processes and correct any obvious errors.

 

Better Personalisation Based Upon Member Behaviors

The current personalisation options offered by major enterprise platforms are poor. Most people (from regions that share the same primary language) see the same view of the community as everyone else. This is because most major enterprise communities rely upon a forum format which really hasn’t changed much in 20+ years.

However, this also means members see a lot of content they have no interest in. However, social media platforms have shifted the trend towards more ‘feed-based’ formats which are personalized to the member-based upon a combination of:

a) What’s broadly popular on the platform today.
b) What’s popular amongst friends/connections of the visitor.
c) What’s likely to be popular based upon past member behaviors/interests/what we know about the member.

For example, if you know what products members own or have declared their interest in previously, only relevant categories, content, and discussions should appear for them on the homepage. They can still find other content if they want, but members should have a most customized experience.

Likewise, if you know the kinds of discussions newcomers visit when they join a community, they should be shown to future newcomers etc…This level of personalisation should be something to be planned for in the intermediate horizon.

 

Develop A Custom Community Experience

It’s pretty common for a group of community professionals to agree that community platforms are terrible and getting worse. This isn’t really true (although they have fallen far behind social platforms in user experience).

The problem is the needs of clients are diverging from one another. Large companies increasingly have diverging needs. One recent client would only consider vendors which met stringently high standards in the core web vitals. Another wanted the ability to run competitions with lucky draws from within the platform itself.

Satisfying emerging needs is nothing new. It’s what we expect vendors to do to stay relevant. The problem is the needs of enterprise seem to be diverging. The point of SaaS companies is they (ideally) make one version of their product which they can sell repeatedly at a high margin. They generally don’t want to work hard to customize a platform to solve one vendor’s unique problem. That’s not scalable nor sustainable.

Some organizations tackle this by taking a platform and spending a fortune on customisations. But they become a pain in themselves. There are three key approaches to resolving this.

The first is to build an entire community experience from scratch. Apple replaced this Jive experience with a (strikingly similar) custom build. DigitalOcean also developed their entire custom community experience.

Another option is to utilize an array of platforms under a broader umbrella. SAP is one example of this. Snowflake, shown here, is another example.

snowflake community example

Another growing trend, as shown by Notion, is simply encouraging people to create, join, and participate in groups that already exist.

notion community example

There isn’t a single answer, but I’d be surprised if we don’t see more organizations taking a different approach to their community experience – one where the hosted platform (if there is one) is just one part of a broader experience.

 

Tokens and Metaverse (Web3)

Thus far most web3 activities have been more hype than substance.

As the web3 hits the trough of disillusionment, we’re hopefully going to see the hype fade and some practical applications emerge. What they are remains to be seen. I don’t imagine any community will move to the Metaverse for now. However, I wouldn’t be surprised to see greater use of digital currencies within some specific kinds of communities.

I can also imagine some communities utilizing tokens they can accumulate through positive contributions to a community which affords them voting rights on the direction and key decisions of the community. I can’t see them being widespread, but in certain technical communities they could be powerful.

 

Next Steps

Predicting how macro-trends will affect community technology is similar to making predictions about how outside forces will shape any industry. We can never be quite sure.

It’s clear that some trends are already in motion (data security on platform selection, losing members to habitual platforms, declining differentiation etc…), others are going to be felt in the more distant future (AI, automations, and tokens).

What’s important is you’re building a community experience prepared to navigate these trends internally and externally. We might be wrong about the timeline, but we’re unlikely to be wrong about the broad direction community technology is going.

The danger is offering a community experience that falls increasingly behind standards to the point where its continuing existence is in peril. If you build a roadmap and make changes now, you’re likely to avoid that fate.

The key steps I’d recommend at the moment are:

1) Check and validate which trends apply to your situation. Use our template as a start, but be sure to adapt and customize it to your situation.

2) Estimate short, medium, and long-term impact of trends. Begin with changes that you’ve already noticed and grow from there. Use our template from above if you like.

3) Build your community roadmap in three phases. Begin with the quick and inexpensive changes. Begin gaining support for the more challenging changes.

4) Work with a group of peers to support each other. Collaborate with your peers when building roadmaps. Benefit from the expertise of others.

Good luck!

 

Resources

Why You Should Build The Definitive Community For Your Industry

There’s something special about communities built by a brand, but not about the brand.

It’s one thing to build a community for your customers.

It’s another thing entirely to build a community for your industry (or topic).

While the former can be great for customer support, gathering ideas, and spreading best practices, the latter can position you as the host of an entire industry.

Whereas the customer communities are great for supporting and retaining customers, industry communities directly attract new customers, position a brand where it needs to be seen, and creates a host of new opportunities.

But doing this well requires you to make some extremely difficult decisions.

 

Why Create An Industry Community?

As anyone who’s hosted a party knows, being the host offers a lot of rewards. On an industry-scale, those rewards are multiplied.

You’re connected to everyone that matters. You have industry leaders competing for your attention. You are the gatekeeper and tastemaker of an industry. You get to exert subtle influence, learn from others, and have everyone in the industry reply to your emails.

But the most direct benefit is simple. You get to attract new customers.

When you create a community for an industry rather than about your brand – you’re attracting new audiences rather than serving existing customers.

Yet even attracting new customers reveals only a fraction of the community’s value.

Here’s a breakdown below:

Unlike a branded community, there is no maximum cap on the community’s potential value.

The only limit is the number of people interested in the industry – and if your community does its job really well, you can grow this number too!

 

How Do We Estimate The Value Of This?

This is a little like asking the ROI of a promotional campaign.

The result depends upon the quality of the implementation and a range of factors.

But that doesn’t mean there aren’t some interesting ways to estimate it.

 

Multiple By Value of Leads

You can also work with your sales team to find the typical value of a lead. This usually means dividing the revenue by leads generated to get a broad estimate.

You can also be more precise by working with sales teams to go through the sales funnel and match members up to similar behaviors.

You might also create separate values for inactive (subscribers), moderately active, highly active to create a total value.

 

Multiply By Costs Of Leads

Another way to estimate value is to look at the advertising equivalent cost. How much does it cost you to acquire a customer (or subscriber) through other channels?

If you know getting someone to subscribe to your mailing list (top of the funnel) usually costs you $100, you might estimate the present value of your 20k strong community at $2m.

Once you begin estimating value using these proxy metrics, you begin to see why it often makes sense for organizations to simply acquire communities. If a community in the SaaS field has 20k members and the average SaaS brand spends $344 to generate a lead, any acquisition value under $7m begins to look like a bargain.

Essentially acquiring a community acts as a short-cut to the lengthy and costly process of acquiring the audience through other channels.

 

Numbers Don’t Capture The Full Value

I’m struck by this quote from Max Altschuler, founder of the SalesHacker community (acquired by Outreach).

“We also use it [SalesHacker] as a value drive in deal cycles. We sell to Sales, so offering a CRO/VP Sales a spot on a podcast for a brand they know that gets 20,000 downloads is a great value driver no other sales vendor can provide and it opens up that relationship at the exec level now.”

There is huge value in being able to feature a prospective client in a community in front of tens of thousands of others. It won’t work for every community but it shows the raw numbers don’t even capture a slither of the full value of the community.

This also highlights something interesting – there is simply natural, innate, immeasurable value from being the provider of the biggest community in your space?

So, why isn’t every brand jumping to do this?

Because creating an industry community requires you to do something really painful – hide your brand name.

 

Don’t Name The Community After Yourself

With a few exceptions (HubSpot), in an industry community you can’t have your name prominently splashed anywhere.

The moment you call this ‘The [brand] community’ – two things happen:

1) People will ask questions about the brand. You will accidentally create a customer support community.

2) Customers of rival products will stay away. Why would they join? They use a competitor brand. This typically reduces your potential audience by 80%+.

It’s very, very, hard to defy these two outcomes if you name this community after organization. This usually causes some consternation amongst marketing and PR professionals who want to see their brand prominently displayed (and avoiding the effort of promoting two brands).

But this doesn’t mean your name can’t appear anywhere…

 

Won’t Members Get Upset If They Find Out It’s Owned By A Brand?

Sure they will…if you’ve kept it a secret.

If you’ve been visiting and supporting an independent bookstore for a few years and then suddenly discovered it was secretly owned by Amazon – you’re not going to be happy.

So don’t keep it a secret!

Be honest and upfront about your involvement and intentions from day one.

I’d suggest having your logo on the website such as (provided by [brand]) while giving the community its own unique name and identity.

This name should be something which reflects a commonly used unique symbol by the people you’re trying to reach.

I’d also suggest in your welcome message and about page that you’re clear about why you’re hosting the community. If you’re hoping to attract new customers then say so. But create rules about how and why this will happen. It might also help to create rules about how the brand engages with the community.

For example:

“While we hope to benefit from identifying potential customers or getting insights from members, we won’t send out promotional emails to community members. Discussions about competitor products will be encouraged – never removed.”

You can build trust and associate your brand name with the community by being honest and upfront from the very beginning.

 

What Do We Need To Get Started?

To keep it simple. There are three obvious things you need to get started.

1) A potentially large audience with a unique set of needs you can serve.
2) A dedicated platform to host your community.
3) A community manager to run the community.

As you can see above, if you don’t have someone in charge, you’re risking anarchy.

If you’re lacking a platform, you have a decentralized social media experience.

If you’re lacking an audience, you have a ghost town.

 

How Do We Get Started? What Should Be In The Community?

How you get started often depends on the size of your audience.

If you’re beginning with zero, it’s best to go through the CHIP process with blog posts, organizing events, engaging with existing groups, and building up your audience until you get up to 100 subscribers/followers.

Once you get near 100 active followers, then you might want to consider having a platform where members can engage directly with one another (hashtags can also work).

As you reach around 200+ members, you probably want to directly promote the community internally to any existing audiences, consider investing in paid promotion, and, eventually, working with influencers and other partners.

Once you have more than 2k members, usually word of mouth and search traffic drives most of your audience (you should optimize for both).

You can see this here:

What Platform Should We Use?

The platform you use primarily depends on your budget and the audience you’re starting with.

When you have no people (and no budget), you usually need to use email.

When you have 10 to 250 you can usually use a WhatsApp group.

At 250 to 1000, you’re looking at Telegram, Email, Circle, and other lists.

Once you get to the upper end of that, it starts to become a chaotic mess. So you need to look at an enterprise (often forum-based) platform which allows for more structured discussions. There is always a slight tension between structure and ease of use.

It’s notable however that many of the most successful industry communities over the years have been built on custom platforms. This provides more flexibility and (often) proprietary technology which may increase a future acquisition price.

 

How Much Will It Cost?

Like most things, it depends.

Many of the big communities acquired by others over the years were launched on a shoestring budget – often by an amateur doing it in their spare time. Others were launched by big brands to much fanfare.

However, we can add a very broad cost estimate to some traditional approaches.

While it’s obvious there are some benefits of the amateur approach (testing things at small-scale first and adapting quickly), you’re less likely to be able to use that approach if you’re working on behalf of an organization.

Organizations have legal, moral, and reputational responsibilities which make them less likely to ‘throw something up and see how it goes’. You can’t launch something which doesn’t look or function well and hope you improve it later (the way a passionate amateur can).

 

Examples of Successful Industry Communities

You can find plenty of successful communities. Each has grown and evolved in different ways. Here are a few common ones.

  • Kaggle. Kaggle launched with a single competition in 2010 for data scientists and gradually added blogs, events, content, and more. Acquired by Google.
  • Element14. Launched by Premier Farnell in June 2009, Element14 is a community for engineers and electronic enthusiasts to chat with one another.
  • Grow, Gain, Retain. Began as a mailing list created by Customer Imperative for customer success professionals before being acquired by HigherLogic.
  • ProjectManagement.com. Began as a place for members to share resources and templates with one another. Over time expanded to enable peer discussions between members. Acquired by Project Management Institute.
  • SalesHacker. Began as a blog before expanding to events, courses, and discussion forums and more. Acquired by Outreach.io in 2018.
  • MindTheProduct. Primarily began as a series of events before expanding to content, courses, and, much later, dedicated community channels in Slack and elsewhere. Acquired by Pendo.
  • RevOps Co-Op. Slack community created by FunneIQ. Began with some content and a Slack channel. Later expanded to resources, videos, interviews, and more.
  • The Instructables. Community for members to share their DIY projects. Acquired by Autodesk in 2011.
  • Honeybook – Rising Tide. Community for small business owners working in the creative economy. Created by HoneyBook, a provider of client management software for small businesses.
  • MakerPad. A definition-stretching community for no-code professionals. Acquired by Zapier.
  • IndieHackers. A thriving Reddit-esque style community for independent professionals to earn money directly. Acquired by Stripe in 2017.
  • CFO Connect. Launched by Spendesk in 2018 is a community for CFOs. The community began by offering events and a few resources later expanded to include a Slack channel, one to one member matching, and more.

Each community took a different approach, but there’s some important commonalities here:

1) They all began relatively simply. They grew the permission asset of their audience over time. They usually focused on doing single things well before expanding beyond that. Over time they expanded to include content, resources, courses, podcasts, discussions, job boards, events, shops etc…

2) They focused on providing the most value to members. At each stage of the journey, the host of the community focused on providing the content, products, and services most in demand by members.

 

The Risk: What If A Competitor Does It First?

Perhaps the biggest reason to do it though isn’t the raw numbers but the fear of competition.

What if a competitor launches a community which becomes the definitive community in the industry before you do?

What if they own that mailing list and the right to contact any of them at any time?

What if they host events for that community where they can bring as many of their staff as they like and sell free sponsorships to themselves etc…

What if they can attract leads for free while you’re still paying hundreds of dollars for each lead?

What if they put together a collaborative steering group which creates defacto industry standards?

Is it worth the risk of not doing it?

 

Conclusion

Being the host of an industry creates plenty of useful benefits. When done well, industry communities can generate leads, convert prospects into customers, gather useful industry insights, and reduce customer acquisition costs.

The catch is you can’t make the community about yourself. If you position your organization’s name too prominently in the community, not only will customers begin to use it to ask product questions, you will also deter customers of competitor brands from joining. You need to give the community a unique name, possibly a unique URL, and let it naturally develop its own identity.

You don’t need to spend a fortune to get started. Many of the most successful industry communities were launched by amateurs in their spare time on a shoestring budget. The secret is to begin by doing the smallest possible things extremely well and expanding gradually. There are plenty of examples you can follow.

And if you’re having trouble using the above to convince others about the value of the community then consider this – what would happen if your competitors moved first to be the hub of the industry?

 

Resources

How To Set The Right Targets And Build A Great Community Dashboard

Here’s a relatively common story.

A community team is given a goal to achieve. This goal is usually something fairly simple like: “increase engagement by 50% by the end of this year!”.

However, a few months into the year, the engagement metrics haven’t budged.

In fact, the numbers are even beginning to drop slightly. No matter how hard the community team works to improve response rates, time to first response, and improve the platform, the overall engagement metrics simply don’t move.

At the end of the year, the engagement metrics are slightly below what they were the year before. The community team receives a negative performance review. Budgets are cut, team members leave, and the community suffers.

This raises the question, who failed here? It’s a more complicated answer than you might think.

(p.s. if you want the video version of how to set realistic community goals, click here.)

Setting The Right Goals For A Community

Perhaps the best way to begin is by looking at the goals themselves.

They were simply the wrong goals to begin with.

I wrote a whole book about this; engagement is a bad goal. It’s never the best metric to track, it’s simply the easiest. Worse yet, the number of posts, likes, shares, simply feels like a good metric for success.

To realize how wrong this is. I once came across a community manager who skyrocketed engagement overnight by removing the spam filter.

There’s plenty of information about finding the ROI of your community. I won’t rehash the entire topic here. The key thing is your goals should come from discussions with stakeholders and feasibility of behaviors.

1) Stakeholder interviews and analysis. You need to speak to as many stakeholders as possible, understand their unique needs and motivations (you can use this script), and undertake stakeholder mapping (see template) to determine whose needs to prioritize.

2) Needs, desires, and behaviors of members. You need to interview, survey, and study members to determine what they need, what they want, and what they desire. You can learn more about this kind of data here.

During this process we often host a workshop with the data to try to identify the right kind of goals for the community. We tend to set the members’ needs and let stakeholders establish their priorities.

You can see an example of this here:

Using our research and this simple framework, we should be able to identify and prioritize possible goals.

p.s. It’s worth noting this is an idealized approach. The reality is often a lot messier (it’s not unknown for a senior stakeholder to ignore all of this and simply set the goal).

 

Communities Need Really Specific Targets

In client calls, I often ask what the goal of the community is.

The person I’m speaking with can often give a clear and specific answer.

For example:

“The goal of the community is to improve product adoption”.

When I ask what metrics would show success, the answer usually becomes a lot more vague.

Often the answer is “plenty” or “lots”. Or, in the worst-case scenario “we’ll know it when we see it!”.

This leaves the community team in an unfair position. They might achieve a great result only for someone more senior to state “they expected more!”.

We need specific targets we can aim for here. For example:

We want to see 25% of active community members utilizing 2+ services.

Reach 3-month avg. 25% call deflection within two years while maintaining 4.2+ satisfaction score.

Generate leads with a value of $323,440 per quarter for 3 successive quarters.

Increase 3-month member satisfaction by 16% by the end of the year.

The challenge is where do these numbers come from?

 

Don’t Pluck Targets From Thin Air

Far too often numbers are plucked from thin air i.e. a 50% increase!

Why 50%? No-one knows! It’s just a nice round number that sounds good.

This often leads to a community team having goals which are impossible to achieve.

Is a 50% increase in call deflection a good target?

It might be if there was a 40% increase in the past year.

If activity rose by 40% in the past year, it might be. If it fell by 40% in the past year, it probably isn’t.

To begin finding the right target, we need to know our trends.

 

Use Trends To Set Good Community Targets

Targets should be based upon current trends with a range which indicates what great, good, ok, and bad look like.

Sometimes you can do a great job in reversing a downward trend but fail to hit your goals because whomever set the goal didn’t realize the community was in a downward trend.

Let’s use client data for a community which has a goal of answering as many questions as possible and in July 2021 answered 3971 questions.

The company wanted to increase this monthly average by 50% within a year.

But is this realistic?

Well, let’s look at the trendline below rather than pluck numbers from the sky.

forcasted community questions answered

The trendline suggests at current rates that 5150 answered questions is the current expected result (a 30% increase). 50% would be an extremely high result.

But if you look closer, you might notice something important.

Since April 2020, the number of answered questions has plateaued!

Expecting a big increase when the community has plateaued is a big mistake. Using data that stretches back to Jan 2018 doesn’t make sense to set community targets for July 2022.

Instead we can do something clever; we can forecast the number of answered posts just using the past year of data.

If you’re using google sheets, you can use ”=ARRAYFORMULA(FORECAST(39:A56,C27:C38,A27:A38))” to make predictions about the future.

Now the result (as you can see here) is different:

forecasted community questions per month using 1 year of data

Notice now the prediction is of 4500 answered questions per month for July (or about a 13% increase over the year).

 

The Difference Between An Increase And An Improvement

You can improve a metric but still be performing worse than last year.

For example, if you had a 40% increase last year and this year you only get a 10% increase, the numbers will still go up but you’ll be doing worse.

An improvement isn’t about improving the absolute number, it’s about improving beyond the performance achieved the year before.

An easy way to do this is to set a performance improvement (I’d suggest somewhere between 10% and 30% – which should be matched by an increased budget) above and below the trend line.

This is what adding this will look like:

community forecastged questiouns per month

Now we can start setting some rudimentary targets with a 10% performance improvement based upon the previous year of data. This might look like:

  • Anything above 5000 is good
  • Anything between 4000 and 5000 answered questions per month by July 2022 is ‘ok’
  • Anything below 4000 is bad

You can change the upper and lower limits from 10% to any percentage increase (or decrease) you like.

Now you can set targets based upon current trends from the past year of data and can see what a performance increase might look like.

 

Set Targets Where You Have The Most Influence

What If You Can’t Control The Outcome?

The biggest problem with using the ‘number of. answered questions’ as a goal (and pretty much any engagement target), is that it’s primarily driven by how many people have a question in the first place.

You can’t exercise much control over that.

If your company acquires more customers (or loses customers), that number will rise and fall through no fault or achievement of your own.

Worse yet, many activity-based metrics have a natural curve over time. As you begin answering most questions, people no longer need to ask as many and engagement drops. This is a good result masquerading as a bad outcome on your stats.

So we need to find out what impact you have.

 

Track These Three Metrics To Identify Your Impact

We want to know how the community compares against other channels.

If, for example, the number of support tickets (or customer support calls) falls by 10% and the number of questions in the community drops by 10%, that’s probably not the community’s fault.

In most cases, we usually want to get the following data:

  • No. questions asked in other support channels vs. in the community.
  • No. visits to the company website vs. the community website.
  • No. new customers each month vs. new community registrations.

Then we look to how closely correlated these are with whichever metrics we’re tracking.

For example, look at the graph below.

commujnity web traffic data

We see historically there is a close relationship between an increase in web traffic and answered questions.

If the web traffic suddenly rises or falls, we would expect community participation to rise and fall regardless of how good a job the community team is doing.

The increase in web traffic above should mean a lot more people are now visiting the community. We therefore need to have a model which dynamically updates the forecasts based upon this relationship.

We now use the same FORECAST function to predict this and show what a 10% above or below the predicted line looks like.

community web traffic trends

You can see now how a big increase in web traffic naturally raises the number of answered questions we should anticipate within the community.

This also raises the expected answer rate. Anything above 6198 by July 2022 is now good and anything below 5071 is a poor result.

This is a very simple explanation of how to set targets. In practice, it can become far more complex. What matters however is now the community team has clear targets based upon actual data which is within their control!

 

How Do You Use Data To Achieve Your Targets

Goals Should Change Behavior

There isn’t much point in setting a new goal if you’re going to keep doing what you’ve always done.

The point in establishing community goals is to change your activities to align with that goal.

If your goal relates to growth, then you should be doing more activities which drive growth.

If your goal relates to call deflection, then you should be doing more activities which drive call deflection.

Goals ultimately change priorities. That means you do more of some activities and less of others.

But how do we know which activities drive the outcome?

We first need to calculate which activities have historically been strongly correlated with the outcome.

 

What To Prioritize To Achieve Your Goals

You need two things; a dataset and an informed opinion.

When you have these two things, you can run a multiple regression analysis to determine which variables influence the goal and by how much.

If you don’t know how to do this, find a data person who can help (or reach out to us – we do it for clients).

Let’s use a client example trying to increase member satisfaction.

We ran a multiple regression analysis on a dataset covering 15 variables and discovered the following:

community data analysis

Don’t worry, you don’t need to know what all of that means.

This essentially says there are three statistically significant (and independent) predictors of member satisfaction within the community. Combined they account for 86% of the variability in satisfaction each month.

We use these three predictors as the basis of our strategy.

  • Objective 1: Increase the number of event attendees.
  • Objective 2: Increase the no. MVPs who make at least 1 post per month.
  • Objective 3: Reduce the average time to first response.

Now we repeat the process above to find the right targets for each of these objectives and show what a 10% or 20% performance increase or decrease would look like.

 

Build Your Community Dashboard

It’s obviously important not to keep targets to yourself but to be able to share them widely and let yourself and your entire team stay on track.

We want to know at a glance if the community is on track to achieve its goals or not. If not, we can make rapid changes in our strategy to ensure it is.

Building a dashboard once you have the data isn’t that difficult. You can use Tableau or PowerBI if you want more powerful functionality.

But we’ve kept it simple and built this one below on Google Sheets.

FeverBee community dashboard template

You might want to click to open up the full image.

We can now track progress over time (we’ve added some dummy data to illustrate). This shows where the community is doing well, where it’s not and, most importantly, it’s tracking the metrics which actually matter!

As you get more data, you can see issues early and address them. You can especially see when a number begins to fall behind its predicted target and try to identify what happened each month.

(p.s. It helps to get familiar with ‘conditional’ cell formatting in Google Sheets (or Excel) to create custom rules for what happens when numbers fall above or below a certain range).

 

Let’s Build Out Your Strategy

Once you have the right targets in place it becomes a lot easier to build out the overview of the community strategy.

community strategy example

Now you have all the key elements in place:

1) A clear goal
2) Two clear objectives to achieve the goal above.
3) Specific targets to track progress towards those objectives.
4) A set of strategies each aligned to achieving those results
5) A set of tactics (or initiatives) to execute the strategy.
6) Clearly identified ‘must win’ battles which identify the hard part of each strategy.

Believe me, it’s a lot better to be working on a strategy you know is aligned to achieving specific results you can feel comfortable about being held accountable to. It all begins with setting realistic community goals and the right targets.

 

Summary

Yes, targets should be SMART. But they need to be so much more than that.

Good goals should possess the following attributes:

1) They reflect the unique value of the community to the organization and audience.

2) You should have the majority influence over them.

3) They should be based upon current trends.

4) They should show what a % improvement looks like, not just the increase.

5) They should translate into specific actions you can execute on.

It’s okay to have multiple goals (I wouldn’t recommend more than three). What matters is you have some goals which are translated into specific targets to guide your work.

Try not to rush the process of setting good community goals. It’s worth investing a little more time (or getting outside help!) to get it right.

 

Resources

©2022 FeverBee Limited, Apartment 2410, 251 Southwark Bridge Road, London SE1 6FN England FEVERBEE