Now you need to answer perhaps the most important question. Are the objectives achieving your goals? If your objective was to get existing members to share their best tips within the community, did that achieve the goal of the community?

It’s easier to set objectives from above. It’s far more difficult to determine if you set the right objectives. It’s hard to be critical of your own work to determine whether it was a success.

Measuring Objectives

At this level, you can usually rely upon more standard proxies of success. A CSAT score, for example, is measured by a survey asking members to rate their level of satisfaction with the product on a scale of 1 to 10. A net promoter score (NPS) is measured by asking customers how likely they are to recommend the product to a friend.

You want to measure the score of members via a survey. Then you need to use a survey to measure the score of visitors. You might use a pop-up survey to non-registered visitors to track the latter or email a sample of your total audience. Ideally, you want to measure this over an ongoing period of time (i.e. send the survey out to a different group each month) or have it pop up at random each month. Be careful of the non-response bias:

• Before and after surveys measuring CSAT score of members.
• Before and after surveys measuring CSAT score of visitors.

This will tell you whether customer satisfaction increased. However, it can’t tell you whether this was due to you achieving your objectives or some random factor.

This leads into the second question. Is the change in CSAT score attributable to the community?

Once again, you can look at the correlation between the two scores. You can see the correlation below:

This shows that the goal was achieved to some degree (i.e. the CSAT score increased by a significant amount of both members and non-members). However, the correlation is stronger for non-members than members.

Improve

Now you can figure out what we need to do differently.

• If the correlation is high between objectives and goals, and the goal was achieved, you need to allocate more resources to the community. This might come at the expense of other objectives or simply acquiring more resources from the organization.
• If the correlation is high between objectives and goals, but the goal was not achieved, you need to change the strategy to achieve the objectives. You have the right objectives but the strategy is not succeeding.
• If the correlation is low between objectives and goals, and the goal was achieved, you need to figure out what is working. It is likely that other objectives are succeeding, or your actions are causing another behavior which is increasing customer satisfaction.
• If the correlation is low between objectives and the goal was not achieved, you need to change the objectives. In practice, this often means ending the community efforts as a strategy.

You can go one level further again and use a multiple regression analysis to determine what impact each of the objectives had on the goal and allocate your resources accordingly. This is the best option but requires an understanding of statistics and statistical packages.

Notice the key insights here. These might be hard to extract. If the correlation is much higher for non-members, it suggests that there could be a natural ceiling to the degree to which a community can influence the CSAT score of members.

This might mean the best additional investment might come from getting more people to see the tips in the first place and, thus, a shift in resources to attract newcomers.