Data Secrets

I stole this graph from this paper. It’s a good data set.

Let’s figure out what’s happening here: 

Looking at the data above, what do you think has happened/is happening to the community?

I’d take a guess at the following. There is no context and it’s open to interpretation. Feel free to add your own ideas.

The community was launched and rapidly picked up a following. Within two years they had a huge number of highly active members and hit the maturity phase.

Then they became a victim of their own success. The conversion rate from newcomers to regulars was either low (i.e. newcomers never feel like part of the group), or the turnover was high (existing members didn’t stick around for long). It became impossible for members to feel like part of the group and retain the sense of community spirit they had when they were smaller. Based upon the information below, I’d guess the latter.

Gradually, a smaller number of members became responsible for an increasing amount of activity. This resulted in a very static membership with fewer and fewer new topics to talk about (declining # of new topics).

When you see the data laid out before you, you can spot potential problems far in advance and take steps to resolve them. These were unseen problems. This community had numerous opportunities to resolve the key issues before 2004. Yet they didn’t. They kept doing what made them successful - and it led to their demise.

Two things:

1) COLLECT AND MEASURE YOUR DATA!

2) You have to understand the lifecycle. This is a textbook maturity phase problem. Very common, but not unavoidable.

p.s. It’s worth reading the paper for this conclusions. 

Comments

[Scott Moore]

Hey there, I like your exercise of walking through interpreting data. I think there are good community folk who get intimidated by large tables.

I fostered a community about these same years (2001-2007) and I can vouch that watching your data can help. While it was much smaller than this community, we also had a leveling off about our 4th year. We managed to continue to grow, mainly because I kept monthly data and could see that participation was leveling off, where it was concentrating, and then worked on changing it.

I think your interpretation is a valid one. I haven't read the paper yet (I will this weekend) because I wanted to play the no-context interpretation game too. :)

I see the presented data a little bit differently. Yes, the amount of activity leveled, then dropped off, but this might not be a bad thing. First, a wiki was introduced and, without knowing the potential depth of the topic or the level of expertise of the participants, it seems a fair amount of activity may have shifted to that stored body of knowledge. This might have reduced the need for new topics because topics might be raised as a wiki page or questions that might otherwise be a topic starter were answered by an existing wiki page. Having overall traffic to the different parts of the site would help clarify if potential community members were diverted to other activities.

The next thing that stuck out for me was that number of posts and number of new threads don't decline at nearly the same rate. So I calculated the average number of posts per thread:

YEAR AVG POSTS PER THREAD
2000 14.7
2001 15.8
2002 54.2
2003 80.0
2004 110.2
2005 133.6
2006 221.1
2007 217.9

Wow, while there are a lot fewer participants creating fewer new topics, they seem to be really chewing on them (though these could be epic flame wars). This could still be a deep and lasting community. I'd be interested in your take.

I'm not sure I'm willing to call them a victim of their own success. They may have had a marketing campaign in the early years that attracted the wrong crowd. We don't get to see what the return poster rate (or conversely the churn) is and can only infer it very broadly from the active and new users. If the month to month churn is high, then people weren't sticking from the beginning.

I agree with both your lessons about looking at your data and add that, while it may not be a complete picture, it can give you places where you want to look deeper.

Can I turn this around and ask what measures you think are missing here that would help you develop a more complete picture?

[Becky]

In addition, did you see how many images were uploaded? Looks to me like they are sharing a lot of photos and it has evolved somewhat from what it originally was.