If you want to clean up your community content, begin in the places where community adds unique value.
-> Sign up for our AI-Ready Communities Program before Friday, November 13 using the code EARLYBIRD to get a $250 discount. Enroll now.
Most Teams Optimize The Wrong Knowledge
The most common approach for organizations that begin cleaning up their community is to begin with the most popular threads.
This usually means sorting the discussions by the number of views, starting at the top and working your way down. If you have the resources to do the top 5%, you do the top 5%. If it’s the top 10%, then it’s the top 10%, etc.
The general mindset is that the threads that get the most visits are the most important ones to optimize.
And this is also the wrong approach.
The most popular threads in most communities are usually some form of:
- How do I set this up?
- What does this setting do?
- Where do I find this feature?
This means the most popular threads in almost every community are the beginner-level ones best answered by existing documentation.
The answer already exists, but people either don’t know what to search for or are too lazy to find it.
Notably, these are the most popular threads in every community and, in percentage terms, have seen the biggest declines in views over the past two years. One client saw traffic to their most popular thread (ominously titled “How do I unsubscribe from ”) collapse by 99.2%.
And the reason is obvious. These are precisely the pages for which a documented answer already exists. LLMs now find that answer and display it when someone searches for that problem. The extra duplicate in the community (using slightly different language, etc.) might’ve had value in the past, but its value drops precipitously in the age of AI.
(Aside – The other problem with views is that organizations have an annoying tendency to export lifetime views, which favor older threads, instead of exporting views over the past three months, which better reflects current popularity).
What Is The Best Knowledge To Optimize In A Community?
The most important knowledge to optimize and create is knowledge that people can only get from a community.
That’s clearly not the fact-based questions you’ll find in documentation. It’s the questions that emerge when people experience products and services and have questions about those experiences.
It’s when customers encounter a problem product teams never anticipated. And when customers need to collaborate with the tiny handful of other customers who have experienced the identical issue to solve it.
This is knowledge that emerges from real experiences, often highlighting edge cases or recent changes. It’s the kind of knowledge that shows the real power of a community in action.
Take my favorite example below:
[Note: It’s sad to see Apple closing their forums on Nov 20]
I’m willing to bet good money the Apple product team has no documented solution for what to do when “Pump Up the Jam” randomly starts playing.
It’s an edge case, one of the most extreme edge cases we can imagine.
I’m willing to bet equally good money that he’s not the first Apple user to experience music randomly playing in the background that won’t stop until it restarts.
That feels like a problem that, if a handful of people ask a question, an LLM is likely to stitch together the answers into a coherent response to everyone – regardless of how many views each receives. It’s the kind of question which should be properly optimized.
And this is just one of tens of thousands of edge cases. Different customers encounter new and unique edge cases every day. They and thousands of others help each other tackle the long tail of customer problems.
The Long Tail Of Community-Generated Knowledge
As you can see above, yes, absolutely, a relatively small number of questions cover a large % of all customer questions. They get the most views for that very reason. They’re the issues most people run into. That’s the head of the long tail at work.
But these are the questions with definitive answers or setup questions. No matter how much you curate/optimize this, any self-respecting LLM will always prefer documentation (though potentially using community knowledge for additional context).
The most important questions to optimize in a community are those where the community adds unique value.
In most of these cases, it’s where a community member stumbled across a unique edge case, an experiential question, or an immediate issue. And, ideally, where members and/or staff collaborated to find a solution. That’s the real value of a community.
These are the discussions we want to focus on because they will offer the most value going forward.
You can see the difference below:
The temptation is to focus on optimizing the top 10% of discussions overall – which will inevitably be on the left. But what you should be doing is optimizing the top 10% of discussions in the three areas on the right.
That’s unique community-generated knowledge!
And even in smaller communities, there are a lot of these questions.
The Unique Knowledge Filter
Before you curate anything, I recommend running your community content through a ‘unique knowledge’ filter.
This should do two things:
- Filter for unique knowledge first. We sort community questions into their unique categories (canonical, procedural, experiential, contextual, ephemeral, etc.). Then filter out those where the community doesn’t add much value, and focus on the three areas where it does.
- Then rank inside that set. Then, within the latter three categories, we filter by views over the past three months and aim to curate the top 10% within that.
At the moment, many community teams focus only on the top discussions. They invest all their time and energy into the most popular areas, which are already well covered by documentation.
Our unique knowledge approach flips this order. First, we decide where the community adds the most value, then filter the most popular discussions within those areas.
If you want to learn how to do this, then sign up for the AI-Ready Communities program.
Learn How To Do This For Yourself
We’ve been helping clients tackle this problem for almost two years. Now we also teach you to do it yourself in our AI-Ready Communities Program.
The program runs live every Monday from 23 November to 14 December (5pm UK / 12pm New York / 9am San Francisco). Each session lasts an hour and is recorded, and the prep material opens on Monday 16 November.
We work through the five critical areas to prepare your community for AI:
- Duplication and contradiction. How to find duplicate and contradictory answers in your community and resolve them.
- Decay and freshness. How to spot content that has gone out of date and add the freshness signals AI relies on.
- Authority. How to decide which answers are authoritative, so AI knows which answer to trust.
- Taxonomy. How to standardize your categories and tags so content is easy to find and classify.
- Terminology and keeping it clean. How to make sure the same thing is called the same thing everywhere, and how to stop new mess building up.
The fee is $1,750 per organization, so bring as many colleagues as you like. Sign up before Friday, November 13 using the code EARLYBIRD and you get a $250 discount.