To Thread Or Not To Thread

Spend 20 minutes reading this paper.

“Using interrupted time series analysis and regression discontinuity design, we observe an abrupt and significant increase in social reciprocity after the adoption of a threaded interface.”

In short, adding threaded discussions (expanded all the way, i.e. you could have comments indented significantly as people respond to middle comments in the discussion), yielded an increase in responses.

Almost all community platforms today offer threaded discussions. Your mileage will vary, but testing flat, threaded, one-deep, and full branch structures of discussions is a low-cost and potentially high-reward endeavour.

Comments

  1. remah says:

    Thanks for an interesting read though it is somewhat denser than most of your references. :thinking:

    My summary is that a hierarchical/threaded interface makes it easier for people to reply to people who have replied to them. As the paper says:

    Once the hierarchical view is active, users behave significantly more reciprocally and tend to engage more in dialogues.

    Social reciprocity is the key term. The researchers identified that another generative model fails to pick up on this:

    We postulate that the original model fails to capture precisely that commenting behavior tends to be reciprocal, i.e. users tend to reply to (sic) comments that are replies to their previous comments.

    So they create a model to better predict how topics in a hierachical (threaded) interface are more likely to break up into highly reciprocal threads than in a simple linear interface. Users can more easily carry on their own reciprocal/bilateral conversation thread where more replies are generated (popularity is the number of replies) over a longer period (increased novelty).

    Note that the early first few exchanges in a topic did not significantly change with the new interface. That is to be expected because replies to the original post don't involve much reciprocity. It takes a while for the topic to breakdown into reciprocal exchanges.

    Sadly, the paper doesn't address issues that I would be more interested in such as:

    • the relevance of responses to the original topic
    • the quality of discussion
    • other response characteristics such as length and formality/informality
    • the popularity of the responses among other users

    I'm particularly interested in Discourse and it's peculiar interface but the researchers don't really distinguish between the hierarchical (threaded) interface they used and a non-hierarchical interface in any detail.

  2. Sarah Hawk says:

    I wonder how much of an impact notification settings have on this.

    While Discourse has an unconventional way of threading I like that I can choose to be notified only when someone replies specifically to me in a discussion, especially in cases where the discussion branches into multiple topics. In those cases the lack of threading makes things very hard to follow.

    I'm also curious about the impact that threading is having on Slack. It seems to be polarising. I hate it but know of others that can't live without it.

  3. Pablo Aragon says:

    Hi,

    I am Pablo Aragón, first author of the aforementioned research article.

    First, I have to thank you all for your interest in reading our study :slight_smile:

    Your comments are very valuable for us, in particular:

    • "the paper doesn't address (...) the relevance of responses to the original topic"
      True, we might extend the model to take other data like positive/negative votes as a proxy of how responses are relevant for the community and how this might be affected by the conversation interface.

    • "the paper doesn't address (...) the quality of discussion"
      True, how would you quantify this dimension...? (it seems a non trivial challenge to me)

    • "the paper doesn't address (...) other response characteristics such as length and formality/informality"
      True, we were interested in language-independent approaches for two reasons: a) the methodology can be easily replicated in any other online forum and b) I am a bit skeptical with the realiability of natural language processing techniques... However, I agree with you and we must address linguistic issues at some point of this research.

    • "the paper doesn't address (...) the popularity of the responses among other users"
      True, it seems related to the "relevance of responses" mentioned above. How would you distinguish between relevance and popularity...?

    • "While Discourse has an unconventional way of threading I like that I can choose to be notified only when someone replies specifically to me in a discussion"
      True, as external researchers we don't have access to data about notifications to users when their commments are replied. Therefore, this is one of the main limitations of our study (which I do not know how to face unless we have access to server logs from an online forum).

    • "I'm also curious about the impact that threading is having on Slack. It seems to be polarising.
      I hate it but know of others that can't live without it."
      I'm curious too :slight_smile: Regarding polarization and online discussion platforms, I think you might find this new approach of interest https://blog.pol.is/pol-is-in-taiwan-da7570d372b5

    Again, thank you very much for your feedback

    Please let me know any further comment/idea :slight_smile:

    Best regards

    P.S. Sorry for replying to every issue in one single message instead of using conversation threading :wink:

  4. Sarah Hawk says:

    Ha. Good to hear from you Pablo. Thanks for jumping into the discussion.

    Discourse tip: you can quote text by highlighting it and clicking the Quote modal that pops up.

  5. remah says:

    Thanks for responding in one message. We can use selective quoting to start a new topic if we want to continue the discussion on one particular issue.

    Actually, my list of issues were not specific issues with, or criticisms of, your paper. They are questions I am more interested in so they don't deserve separate responses in this topic.

    I really appreciated your paper because it shows the value of identifying reciprocal conversations. If you had also addressed any of the issues I mentioned then they would have obscured that important idea.

    Now that I've had time to reflect on your paper, I think it is far more useful than I initially thought.

    Your paper clearly shows how reciprocal conversations can make forum statistics look a lot better. But my unspoken question was are these reciprocal conversations beneficial? In other words, I assumed that more participants is better.

    In my mind, I was sceptical that reciprocal conversation is actually good discussion. I originally saw the "break up" into highly-reciprocal threads as a "breakdown" of positive discussion. Fortunately I used the more positive phrase when I posted:

    I had originally imagined two people in a conversation meandering off-topic with a lot of mutual ego-stroking. But I have changed my mind.

    I now think that the vast majority of reciprocal conversations will be answering questions, resolving problems, focussed on the original issue or a relevant side issue, developing ideas in more detail, or of positive benefit to the feeling of community (e.g. as is happening right now in this forum with a conversation about tap dancing).

    Now I can properly thank you for your very good work. :tada:

Make your own comment on this post at FeverBee Experts

©2017 FeverBee Limited, 1314 New Providence Wharf, London, United Kingdom E14 9PJ FEVERBEE

We’d like you to join us.

We’re a group of community professionals who push each other to the cutting edge of our work. Every day we share our experiences, highlight new expertise, and push one another to become the best community builders we can be.

The day you join, you’ll get access to 1000+ practical tips, dozens of resources, and unlimited access to the top experts in our space. You’ll also get to see how we go about building a community amongst our own audience.

It takes just 30 seconds and doesn’t cost a thing.

CLICK HERE AND JOIN FEVERBEE

LevelPrice 
ExpertThe price for membership is $0.00 now.
Select
EnterpriseThe price for membership is $940.00 per month or $8,400.00 per year
Select
Large EnterpriseThe price for membership is $7,450.00 per month or $68,000.00 per year
Select