This abstract reminded me of a recent project.
Over the summer, we worked with a community team that spent just over 40% of their time creating content for members.
They were creating videos, webinars, playbooks, announcements, writing personal blog posts, etc…
But they hadn’t ever checked which content was most helpful to members. They simply kept pumping out more of it.
We suspected this was overkill but needed the data to prove it.
So we ran three simple analyses.
The first was a simple regression analysis. We clustered the content into groups by the medium (videos, webinars etc..) and looked to see which type of content received the most helpful votes.
Next, we did the same analysis but clustered the content by topic.
Finally, we interviewed a handful of community members and asked them to recall which content they had found most useful within the past few years.
The first analysis showed playbooks were by far the most popular type of content. Announcements scored dead-last (despite the team having to spend the most amount of time jumping through numerous internal hurdles to post these).
The second showed newcomer level content received slightly more helpful votes than any other type.
The interviews, however, yielded the most fascinating insight; community members couldn’t keep up. They would bookmark material which seemed really useful but never get around to reading it. This made them feel bad.
In short, the effort the community team was putting into creating lots of fresh content was making members feel bad.
We ran a follow-up analysis and discovered the number of helpful votes had declined in recent years in an almost inverse proportion to the quantity of content created.
We then used this data to drastically cut down on the quantity of content created, stop publishing announcements, and double-down on the playbooks and newcomer-level material.
This kind of analysis isn’t especially difficult to do and can help you save time while giving members a better experience.
