How to Calculate the ROI of Online Communities

By Richard Millington

Chapter 3  LinkedIn  Google Plus
ROI People

The formulae to calculate return, profit or ROI are simple to understand. Yet actually measuring the value of communities in any of these three forms is difficult. This is because measuring value means confronting five major challenges.

1. Data Collection

By far the most difficult challenge is collecting the necessary data to measure the value generated by communities. The data required to measure value is often not collected. There are several reasons for this.

The first is the difficulty in identifying what needs to be measured. Very few platforms come with an inbuilt system for measuring the ROI. This means it is left to the community professional to decide which metrics to collect. These metrics are often more closely related to activity (or key performance indicators) than to value created. For example, very few community professionals collect data on spending levels and compare this with levels of activity in the community. Yet, such data could be critical in order to measure the value of a community.

Few platforms have inbuilt ROI measurement. It's left to the community pro to decide which metrics to collect.

 

Click to Tweet

Sometimes, missing data makes comparisons very difficult. Unless it is possible to benchmark current spending when members join, it might prove difficult to make comparisons a year later.

The second is obtaining the data once it has been identified. Very often, sales figures, salary data, overheads, and even historic CSAT metrics or NPS scores are difficult to obtain. This can be because the organization is reluctant to hand this data over to a community professional or there are other restrictions. For example, increasing data privacy laws limits the number of surveys that can be undertaken of members and the type of data that can legally be collected from members in many regions of the world.

The third is a time challenge. Identifying, collating, and tracking this data often takes considerable time. This is time that cannot be spent on increasing the engagement metrics, which are often the very metrics the community professional has used to justify their value in the past.

2. Determining what value means (lean vs. six sigma)

It’s often difficult to assign community benefits easily into a top line (increasing revenue) or bottom line (reducing costs) framework.

For example, imagine WidgetCorp launches a community for their widget creators around the world to share best practices. As a result of these best practices, the average worker increases their productivity from creating 100 widgets an hour to 150 widgets an hour. Does this mean you now have 50% more widgets to sell at increased revenue? Or does it mean you can reduce your widget creation costs by 50%?

Let’s take a more applicable example. Imagine a community is designed to generate leads. The community generates 500 leads in its first year. Do you calculate the returns generated by those 500 leads, or do you calculate the cost saving of generating those leads via other methods?

This challenge (often referred to as the lean vs. six sigma debate) means that a benefit such as customer acquisition can be subjectively included as a cost saving or revenue generated. This depends whether the organization would have acquired that customer via either channel. Other benefits are hard to categorize in either (e.g. reduced time to market based upon new product ideas).

A further challenge is whether to include common objectives which are a layer or more removed from a direct ROI benefit. For example, customer satisfaction scores are clearly important. Yet, unless satisfaction is directly connected to increased revenue, it is difficult to assign it directly as a benefit.

For the purposes of this project, we have decided to embrace a flexible approach designed to encompass the most commonly cited benefits within broad categories instead of a rigid approach, which would eliminate common goals but would not be directly connected to an improved ROI.

3. Measuring And Attributing Impact

The third challenge is measuring and attributing impact to the community. This is most commonly determined by comparing trend data, either before and after the community was created, or before and after a group of members joined the community. For example, if members join the community and their purchases increase, this is a measurable change in member behavior. However, correlation is not causation.

Attribution is how much of the measurable impact is the result of (or attributable to) the community.

 

Click to Tweet

The biggest challenge is not measuring change but proving attribution. Attribution is how much of the measurable impact is the result of (or attributable to) the community. For example, customer satisfaction might rise as a result of community participation, or it might have increased due to investment in the performance or quality of the product and service.

A company (e.g. Apple) might release a major new product or undertake an intensive marketing campaign during the same time period. As a result, their customers both inside and outside of the community are likely to increase their spending. If we only measure behavior of members in the community, we will see a spike in sales which we might wrongly attribute to the community.

Attribution also helps us to identify benefits when there has not been a measurable impact. For example, imagine the customer satisfaction score among community members hasn’t changed, but the score among non-members has plummeted during the same period. This might be huge value as a result of the community, which otherwise would not have been noticed.

4. Conversion and Generalisation

The fourth challenge is generalization. This is whether the data gathered so far can be generalized across a broad group to create a total figure. To understand generalization, we need to understand measurement. Very often, it is impossible to collect a complete data set. Instead, we measure the behavior, sentiment, or buying habits of a sample of members.

Often it's impossible to collect a complete data set, so we have to measure across a sample of members.

 

Click to Tweet

This happens in polling, too. Pollsters do not poll an entire country. They poll a sample which they hope best reflects the entire country.

We might determine, for example, that spending of this sample has increased by $20 compared with non-members over a specific time span. However, does this mean that we can multiply this $20 by the total number of members to determine the total value generated by the community?

This depends entirely on whether the sample is representative of the group as a whole. Very often, this is not the case. For example, imagine you use a survey to collect responses on buying behavior. You then multiply the difference in value by the total number of members. You have just committed a major generalization error.

First, a large percentage of those members might no longer visit the community. Can you really claim someone who no longer visits the community is still increasing their buying behavior as a result?

Second, the survey might incur a response bias. This happens when the most active members (those who purchase the most products) are most likely to complete the survey. This means it is not representative of the group as a whole. We can combat this by developing several representative groups and combining them (e.g. most active members, lurkers, etc.), or by ensuring the survey uses a quota system reflective of the entire community.

If 90% of active members lurk and don’t participate, then the responses of this group should comprise 90% of the results. The surveys you collect should also be weighted to reflect this breakdown.

However, this also highlights another major problem. How do you determine relevant quotas? Is it by level of activity? By gender? By age? By type of products purchased, etc? How will the results vary by the different samples you collect?

5. Average Gross Margin

The fifth, and least understood, challenge in measuring the return is calculating the average gross margin. It is relatively easy to calculate whether a customer’s spending has increased or decreased. It is difficult to identify whether this generates any additional profit for you.

For example, a customer sees members talking positively about your brand’s new television and pays $1000 to purchase it. If we’re only tracking increased spending, we would attribute that $1000 as value created by the community. But this is a mistake.

The television costs money to make and sell. It doesn’t matter what the product costs to buy; what does matter is what the product costs to make and sell. The difference between them is the gross margin of that product.

If the television has a 30% gross profit margin, the profit would be $300. This $300 is known as the contribution or gross margin. This is the value created by the customer buying that television. This is what we need to measure.

Unfortunately, the revenue generated by different products and services will vary significantly. For example, an organization might host events, sell products, and a consultancy service. An event might be a loss-leader to increase consultancy revenue. However, if the community increases spending on only the events, it might be increasing costs but not generating any profit.

If a community increases spending on only events, it might be increasing costs but not generating any profit.

 

Click to Tweet

A common proxy metric here is to use the profit margin of the entire organization. Yet, this is a mistake. The profit margin already includes a variety of long-term costs which are not related to the cost of selling a single extra unit (the office, etc.). The contribution margin is the difference between the revenue generated by the item and the cost of selling a single additional unit.

Combined, these five challenges add existing layers of complexity to an already complex and complicated task of determining value and ROI. The methods we embrace to calculate the ROI of online communities must navigate through these challenges. These challenges also identify the process we take to determine the ROI. First, we collect the data; second, we determine the type of value; third, we measure the impact; fourth, we determine attribution; and fifth, we multiply (where relevant) by the average gross margin.

Summary

  1. It’s hard to collect the data we need to measure communities. This can be for internal reasons, data protection laws, or sheer inconvenience.
  2. We need to determine if we’re measuring costs saved (lean) or additional value generated (six sigma).
  3. We need to include attribution, not just impact. Attribution usually requires comparison.
  4. If we’re measuring a sample, the sample has to reflect the composition of the community. Do not include non-visitors within your sample if they are not reflected within the survey data.
  5. We must multiply any revenue generated by the average gross margin. Many forget this step.

Chapters

Share

 LinkedIn LinkedIn

 Google Plus Google+