Good Answers Can Also Be Unhelpful

April 23, 2017 Comments Off on Good Answers Can Also Be Unhelpful

Many communities today are moving towards simple rating systems in responses.

Readers can select whether a response is helpful or not helpful. Some are measured by this metric (it’s as good a metric as any).

But what if a member asks a question you can answer but not resolve?

What if a member wants a product you no longer sell? Or to request a feature you can’t create?

You can provide the most thoughtful, detailed, answer possible and it’s inevitably going to receive a downvote (or no votes) simply because it didn’t resolve the question.

Ouch!

This becomes an especially big problem during new product launches/changes when people have a lot of questions which can’t easily be solved.

If you’re measured by satisfaction scores (or % of helpful votes), your success will fluctuate randomly regardless of what you do.

This means two things:

  1. You need to tag and exclude the questions which were impossible to resolve in the first place. Be aware of the temptation to tag questions which are likely to be unpopular here too.
     
  2. You need to take a random sample of these scores and compare them. If you don’t want to tag every question, use samples. Take a random sample (>30) of votes on unsolvable questions and unsolvable questions. Compare the two and check for significance. Now you can make relatively safe compensations for drag caused by these questions.

It’s a relatively minor problem unless you’re measured by member satisfaction. In which case it can suddenly become a very big problem indeed.

Comment on this post at FeverBee Experts

©2018 FeverBee Limited, 1314 New Providence Wharf, London, United Kingdom E14 9PJ FEVERBEE

We’d like you to join us.

We’re a group of community professionals who push each other to the cutting edge of our work. Every day we share our experiences, highlight new expertise, and push one another to become the best community builders we can be.

The day you join, you’ll get access to 1000+ practical tips, dozens of resources, and unlimited access to the top experts in our space. You’ll also get to see how we go about building a community amongst our own audience.

It takes just 30 seconds and doesn’t cost a thing.

CLICK HERE AND JOIN FEVERBEE

Level Price  
ALL COURSES The price for membership is $750.00 per month or $7,200.00 per year
Select
Community Development Program The price for membership is $1,100.00 per month or $10,560.00 per year
Select
Director of Community The price for membership is $8,770.00 per month
Select