TL:DR – A legitimate ROI figure is more important than a high ROI figure. Discover how the Microsoft Answers team conservatively estimates $80m in ticket deflection.
The typical call deflection formula gives wildly inflated ROI figures for a community.
That’s partly because those running community programs have no motivation to be conservative. Why would would we? You want to show your community is generating the highest possible return!
But the goal of calculating ROI isn’t to give the highest number it’s to gain and sustain support for a community.
You do that by showing you’ve taken every effort to ensure your number is valid.
This is why you want to be conservative and consider all the arguments against your metric.
You want to pre-answer every ‘but what about…?’ question in the calculation itself. Then you can have discussions about revising the metric up vs. revising it down.
This is how you get to a legitimate figure.
A Breakdown Of Microsoft Answers’ Call Deflection Methodology
Two weeks ago, I hosted Dennis Pollett to share Microsoft’s approach to calculating community ROI.
We’ve been working with the Microsoft Answers team for years and I can confidently state it’s beyond almost any other brand community out there.
Here are the stats:
Traffic: 1.5 billion unique customer sessions per year
New questions: 750,000 per year (2,000 per day)
Languages supported: 24
Response rate: 99%+ of questions get a response
Known answerer contribution: 95–96% of responses come from trusted volunteers, moderators, and paid agents
Following the standard methodology (views to accepted answer * 0.05 * cost of a support ticket), would send the return figure into the billions!
But, as Dennis explains in the webinar, Microsoft does everything possible to be conservative.
I want to share just how conservative this methodology is.
Focuses only on supported content.
This means:
Only using calculations from actively supported content (e.g. Windows 10 vs. Windows 7).
If content can’t deflect a support case (i.e. users can’t open a support ticket), it’s not included.
Regularly archive outdated content using date fields.
Reduces the number of customer volume in calculations
Focus on unique customer sessions vs. pageviews.
Exclude traffic from bots and marketing campaigns.
Only count sessions of AT LEAST 15 seconds.
If a user browses multiple product forums in a single visit, only the last product they landed on gets credit. This avoids duplicate counting.
Calculate success using a clear standard
Ask a binary yes/no question (“did this solve your issue” vs. “was this helpful?”) – helpful gets inflated responses (27% higher!)
If a user interacts with multiple replies before making one solved, only the final success counts.
Embed success ratings in the thread vs. using a pop-up survey (this avoids inflated scores from users who stay longer just to close the poll).
Uses defensible deflection rates
Most organisations use a figure of between 10% and 30% of users who would have opened a support ticket if not for the community. Microsoft uses 0.5% (1 in 200).
This is based on journey tracking – how many users actually went from the community to open a support case (and aligns closely to our study).
Aligns deflection to assisted support costs.
Consumer cases cost $5 to $20 to deflect, commercial cases cost hundreds of dollars. The type of case matters more than a broad average.
You will notice that at every stage of this journey, Dennis could choose between a higher number or a more legitimate number – and at every stage he’s decided to go for a more legitimate number.
I strongly recommend you watch the full session here.
And if you want help calculating the value of your community, our community intelligence team would love to help.
