Good Answers Can Also Be Unhelpful

April 23, 2017 Comments Off on Good Answers Can Also Be Unhelpful

Many communities today are moving towards simple rating systems in responses.

Readers can select whether a response is helpful or not helpful. Some are measured by this metric (it’s as good a metric as any).

But what if a member asks a question you can answer but not resolve?

What if a member wants a product you no longer sell? Or to request a feature you can’t create?

You can provide the most thoughtful, detailed, answer possible and it’s inevitably going to receive a downvote (or no votes) simply because it didn’t resolve the question.


This becomes an especially big problem during new product launches/changes when people have a lot of questions which can’t easily be solved.

If you’re measured by satisfaction scores (or % of helpful votes), your success will fluctuate randomly regardless of what you do.

This means two things:

  1. You need to tag and exclude the questions which were impossible to resolve in the first place. Be aware of the temptation to tag questions which are likely to be unpopular here too.
  2. You need to take a random sample of these scores and compare them. If you don’t want to tag every question, use samples. Take a random sample (>30) of votes on unsolvable questions and unsolvable questions. Compare the two and check for significance. Now you can make relatively safe compensations for drag caused by these questions.

It’s a relatively minor problem unless you’re measured by member satisfaction. In which case it can suddenly become a very big problem indeed.

Comment on this post at FeverBee Experts

©2023 FeverBee Limited, Apartment 2410, 251 Southwark Bridge Road, London SE1 6FN England FEVERBEE

We’d like you to join us.

We’re a group of community professionals who push each other to the cutting edge of our work. Every day we share our experiences, highlight new expertise, and push one another to become the best community builders we can be.

The day you join, you’ll get access to 1000+ practical tips, dozens of resources, and unlimited access to the top experts in our space. You’ll also get to see how we go about building a community amongst our own audience.

It takes just 30 seconds and doesn’t cost a thing.


Level Price  
ALL COURSES $750.00 per Month.
Community Development Program $1,100.00 per Month.
Director of Community $8,770.00 per Month.
Free Trial - Accelerator Program Free.
Community Accelerator (founding members) [annually] $250.00 per Year.
Community Accelerator and FeverBee Intelligence [monthly] $45.00 per Month.
Community Accelerator and FeverBee Intelligence [annually] $450.00 per Year.
Full Community Upgrade (Large Enterprise Only) [monthly] $750.00 per Month.
Full Community Upgrade (Large Enterprise Only) [annually] $7,600.00 per Year.