July 18, 2023

Introduction

  • Social media platforms facilitate conversations between users who might not interact otherwise
  • These interactions can stimulate exciting exchanges or heated disagreements
  • We expand on existing literature by developing sub-graph motifs for Reddit to measure conversational conflict (Coletto, 2017)
  • We can measure conversational conflict with easy to compute features: no NLP necessary

Data Source

  • Reddit: a social network made up of user-created communities
  • Users interact through post and comments, and up-vote or down-vote those posts
  • These votes indicate the value ascribed to the comments or posts by the poster or commenter they reply to
  • This particular data unavailable through Reddit API, since we use the votes of particular users

The UI: Upvote

The UI: Downvote

Methodology

  • Construct sub-graph motifs between two users: User A makes a post or comment, User B replies, and User A votes on the reply
  • Aggregate motifs, calculate the ratio of motifs ending in a down-vote to those ending in an up-vote “contentiousness” by subreddit
  • Compare across subreddits or days to extract insights about where and when conversational conflict takes place

Dataset Overview

  • Analyzed posts from the first two months of 2023
  • Limited to 9,131 subreddits with 1,000+ motifs and 1,000+ members
  • Overall contentiousness score computed for each subreddit
  • Scores range from 2.8 (r/israel_palestine) to 0 (private babybumbs communities)

Observations

  • Top 10 most contentious subreddits: all related to news or politics
  • Least contentious subreddits: built around collaboration or private discussions
  • Most contentious topics: football (soccer), U.S. politics, science, economics, sports
  • Least contentious topics: computing, dogs, celebrities, cycling, gaming

Should you recline your airplane seat?

  • Posts discussing whether it’s okay to recline your airplane seat have a contentiousness of 1.5, while other similarly active posts have a contentiousness of 0.9

Survey Data Results

  • Regressed contentiousness against user assessments of subreddits in Weld, Zhang & Althoff[2021]
  • Controlled FWER at \(.05\) level
  • Across subreddits, 1 SD increase in contentiousness is associated with:
    • 1.5 minute increase in daily time on subreddit
    • .4/10 pt decrease in perceived quality and inclusion
    • .45/10 pt drop in their assessment of how ”included and able to contribute…new and existing members” feel.

Case Study: r/formula1 in 2021

Discussion

  • Detection of changing dynamics helps in monitoring and managing the level of conflict
  • Provides opportunity for users and algorithms to tailor conversational experiences
  • Some level of contentiousness keeps a community vital and honest, avoiding echo chambers
  • One sign of an echo chamber: non-contentious discussion of a contentious topics (i.e. r/stocks (.46) vs r/superstonk (.12))

Conclusion

  • Optimal amount of contentiousness is likely not zero: each community needs a balance
  • Understanding the dynamics of contention can assist in moderating online communities

References

  1. Levy, S., Kraut, R. E., Yu, J. A., Altenburger, K. M., & Wang, Y.-C. (2022). Understanding conflicts in online conversations. Proceedings of the ACM Web Conference 2022, pp. 2592–2602.
  2. Napoles, C., Tetreault, J., Pappu, A., Rosato, E., & Provenzale, B. (2017). Finding good conversations online: The yahoo news annotated comments corpus. Proceedings of the 11th linguistic annotation workshop, pp. 13–23.
  3. Coletto, M., Garimella, K., Gionis, A., & Lucchese, C. (2017). Automatic controversy detection in social media: A content-independent motif-based approach. Online Social Networks and Media, vol. 3, pp. 22–31.
  4. Weld, G., Zhang, A. X., & Althoff, T. (2021). What makes online communities ’better’? Measuring values, consensus, and conflict across thousands of subreddits.