[section]
[aside]
Are We Using The Right Tactics?
Data-Driven Improvements
[/aside]
[article]
Now you have looked at the execution of the tactics, you need to determine if the tactics themselves are helping you fulfil your strategy. This will highlight if you are using the right tactics.
If the tactics have been well-executed but there hasn’t been been a clear change, then you are probably using the wrong tactics to fulfil your strategy.
If the tactics have been badly executed, then it doesn’t really matter what you do. It is likely your results will fall somewhere between the two. This makes measurement a little more difficult (and even more important). This means you need to measure the degree to which the tactics have fulfilled the strategy.
[/article]
[/section]
[section class=”flipped grey”]
[aside]
[/aside]
[article]
Defining Variables To Measure
If you want to know whether the tactics fulfilled the strategy you first need some benchmarks to measure the strategy. This will vary by strategy. For example, your strategy might be to make regular members envious of the top members in the community (we’ll measure the objectives later).
This means the variables to measure might include:
- Change in attitude. You might measure how members feel towards top members today and how they feel later on.
- Change in social norm. You might measure if new social norms initiated or exacerbated by interviews have spread among the group.
- Change in visible behavior. You might measure if top members are mentioned more frequently since they have been interviewed.
All of the above could indicate the tactics have been successful in achieving their goal.
At this stage, you also want to look for a correlation between the measured changed and the desired outcome.
[/article]
[/section]
[section]
[aside]
[tweet text=”If the tactics have been well-executed but there hasn’t been been a clear change, then you are probably using the wrong tactics”]
[/aside]
[article]
Measuring The Impact Of Tactics
Once you know what to measure, you need to know how to measure it. This often means we need to measure before and after outcomes.
- Change in attitude. There are some complex methods to measure attitudes. Some of these are worth exploring. Most likely, however, you want to run a before and after survey asking about attitudes towards top members. You can then compare the before and after to see the impact of the tactics. Another approach might be to run a sentiment analysis and see how people feel about the issue before and after.
- Change in social norm. You might measure whether the tactics have created a sustained new behavior you believe is correlated with the change in attitude. That behavior might be encapsulated by mentions of a specific phrase, a particular type of discussion, or a medium of sharing information.
- Change in visible behavior. You might measure another behavior change that would reflect a change in attitude. If the goal is to provoke envy, what behaviors would suggest it’s working? This might be others also performing that behavior, for example. Or increased positive mentions of the members featured, etc.
This will tell you whether the strategy is being fulfilled. But it will not tell you why something is or is not working.
[/article]
[/section]
[section class=”flipped grey”]
[aside]
[boxout]
Did Tactics Cause The Change?
It’s important here not just to measure whether there was a change during the same period you executed your strategy. You also want to measure whether the change was due to the tactics you executed. You don’t work in isolation. There are many other variables that might influence the attitude change. Your company might have improved its customer service support, or executed a new advertising tactic during the same period.
This means you next need to measure the relationship between the tactics we execute and the changes in behavior.
[/boxout]
[/aside]
[article]
Analyze Why It Works
At this stage, you usually want to run a simple correlation analysis. At the most basic level, this is really simple to run. You use the metrics you previously collected from the tactical analysis (e.g. reach, views) and then the metrics from your survey response to see if they are correlated.
You can perform this using the =CORREL feature in Excel or Google Sheets.
Do they both rise and fall with each other over time?
You then look to see how positively correlated they are. For example, if the combined views of videos, questions asked, or length of time videos were watched rises closely with a change in attitudes or envy of the experts, this might imply a relationship*.
You will also be able to see which tactics are most effective here. For example, which tactics correlate most closely with an increase in whatever proxy metric you are using to measure the strategy. Running a correlation analysis at the most primitive level can be done simply in Excel.
This indicates that both views and viewing time have a moderate and strong correlation with the emotion we are trying to provoke. You could also run a regression analysis to indicate how a likely increase in one variable will impact another, or (ideally) a multiple regression analysis to reveal the influence of each on the envy and any potential overlap between them.
You can now embrace this within your analysis to reveal useful insight.
[/article]
[/section]
[section]
[aside]
[tweet text=”Stop executing tactics with low correlations to the success of the strategy. Invest more time in things with high correlation.”]
[/aside]
[article]
Improve
Now you can start making some useful assumptions about what to spend more or less time on.
You want to stop doing the tactics with low correlations to the success of the strategy and invest more time in things which have a high correlation with a successful strategy. You might also consider which tactics are the best bang for our buck. What tactics have a high correlation with success but do not require a lot of time and money?
Once again, you can start allocating our time and resources to be as effective as they can possible be. There will always be some repetitive tasks you will need to do. But the rest needs to be ruthlessly optimized to achieve the best results.
It’s worth noting here that correlation does not prove causation. To prove correlation, you need to run an experiment and measure the outcomes. However, once you have established correlation, you can track these metrics in future and see if the correlation holds true as you allocate more time to the resources. The more data-points you have, the more accurate your predictions will be.
In the situation above, additional investment in higher-quality video would likely lead to a bigger increase in attitude change. Thus, you might allocate more time and resources to finding a better video editor and improving the production quality of the video.
This is a clear actionable improvement based upon data and a logical conclusion. This is the level you want to be working at. You’re using your data to make effective decisions.
[/article]
[/section]
[summary]
- To measure the success of a tactic, you need to determine if the the emotional state of members has changed. This can be measured by surveys, sentiment analysis, or visible behavior.
- Benchmark the current level before you implement the strategy.
- Determine if there has been a change in your desired direction.
- Run a correlation analysis to identify a possible relationship between metrics you are tracking and the desired change in strategy (regression and multiple regression analyses are even more useful).
- Spend more time in areas with strong correlation and less time in areas with weak correlation.
[/summary]
