There isn’t a clear link between data we collect and actions we take.
We recently discussed metrics in CommunityGeek. Everyone can identify metrics they measure, but no-one could explain how they use this data.
It should work like this. You visit Google Analytics (or Omniture, or whatever) to collect specific data. You don’t browse, you know exactly what you need. You drop the numbers into your model. This model has a decision point. Based upon this, you perform pre-determined actions.
Here are a few examples.
1) 3 month decline in unique, new, visitors = more time on growth channels. You measure the number of unique, new, visitors each month. If it begins to decline in 3 successive months, you spend less time participating in discussions and devote more time to promoting the community. This might mean creating slides of best advice, sharing case studies in external channels, or establishing sustainanable growth methods.
2) 3 months of 30% visits from mobile = a mobile optimzed site. You decide that once mobile visits accounts for 30% of all visits in 3 successive months (not a blip), you will develop a mobile-optimized site. Each month you track the % of visits from mobile. Once the above condition is met, you invest in a mobile site. Again, here data directly affects actions.
3) If new registration to contribution number > 5%, keep intervention. You made an intervention in the welcome e-mail to get more new registrations to make a post. The previous number was 5%. If this has increased, you will keep it, if it has fallen, you will try something different.
A few important points about community measurement.
- Begin with a hypothesis, model, or condition that affects specific actions you take. If you don’t have this, you’re wasting time.
- Don’t confuse health with ROI. Don’t confuse community health metrics (growth, activity, sense of community) with the community ROI metrics (increased revenue/reduced costs).
- Measure what’s important, not easy. What’s important to measure and what’s easy to measure are two different things. To measure the conversion funnel involves pulling data that spans unique visitors * unique new visitors, registrations from member database (different system), systematic sample to see how many made a contribution (manual observations). Not easy, but incredibly important.
- Spend only a few minutes in the analytics package. You shouldn’t spend more than a few minutes in any analytics package. You’re looking for numbers to drop into your model. Nothing more. The moment you begin browsing, your going to find noise, not signal.
Here is the key question. Are you collecting data for your own vanity? Are you collecting data for your own self-validation (or self-preservation)? Or do you have an actual model and method of using the data.
Unless you know exactly what you’re going to do with data, there’s no point in collecting it.
