Probabilistic models of inauthentic behavior in social networks

Team

• Swapneel Mehta (New York University)
• Bogdan State (Scie.nz)
• Richard Bonneau (New York University)
• Jonathan Nagler (New York University)
• Philip Torr (University of Oxford)
• Atılım Güneş Baydin (University of Oxford)

Abstract

Online disinformation is a dynamic and pervasive problem on social networks as evidenced by a spate of public disasters in light of active efforts to combat it. Since the massive amounts of content generated each day on these platforms is impossible to manually curate, ranking and recommendation algorithms are a key apparatus that drive user interactions. However, the vulnerability of ranking and recommendation algorithms to attack from coordinated campaigns spreading misleading information has been established both theoretically and anecdotally. Unfortunately it is unclear how effective countermeasures to disinformation are in practice due to the limited view we have into the operation of such platforms. In such settings, simulations have emerged as a popular technique to study the long-term effects of content ranking and recommendation systems. We develop a multiagent simulation of a popular social network, Reddit, that aligns with the state–action space available to real users based on the platform’s affordances. We collect millions of real-world interactions from Reddit to estimate the network for each user in our dataset and utilise Reddit’s self-described content ranking strategies to compare the impact of coordinated activity on content spread by each strategy. We expect that this will inform the design of robust content distribution systems that are resilient against targeted attacks by groups of malicious actors.

Publications

1. Mehta, Swapneel, Bogdan State, Richard Bonneau, Jonathan Nagler, Philip Torr, and Atılım Güneş Baydin. 2022. “Estimating the Impact of Coordinated Inauthentic Behavior on Content Recommendations in Social Networks.” In AI for Agent-Based Modelling Workshop (AI4ABM) at the International Conference on Machine Learning (ICML) 2022.
   • Abstract
   • Bib

   Online disinformation is a dynamic and pervasive problem on social networks as evidenced by a spate of public disasters in light of active efforts to combat it. Since the massive amounts of content generated each day on these platforms is impossible to manually curate, ranking and recommendation algorithms are a key apparatus that drive user interactions. However, the vulnerability of ranking and recommendation algorithms to attack from coordinated campaigns spreading misleading information has been established both theoretically and anecdotally. Unfortunately it is unclear how effective countermeasures to disinformation are in practice due to the limited view we have into the operation of such platforms. In such settings, simulations have emerged as a popular technique to study the long-term effects of content ranking and recommendation systems. We develop a multiagent simulation of a popular social network, Reddit, that aligns with the state–action space available to real users based on the platform’s affordances. We collect millions of real-world interactions from Reddit to estimate the network for each user in our dataset and utilise Reddit’s self-described content ranking strategies to compare the impact of coordinated activity on content spread by each strategy. We expect that this will inform the design of robust content distribution systems that are resilient against targeted attacks by groups of malicious actors.

   @inproceedings{mehta-2022-inauthentic,
     title = {Estimating the Impact of Coordinated Inauthentic Behavior on Content Recommendations in Social Networks},
     author = {Mehta, Swapneel and State, Bogdan and Bonneau, Richard and Nagler, Jonathan and Torr, Philip and Baydin, {Atılım Güneş}},
     booktitle = {AI for Agent-Based Modelling Workshop (AI4ABM) at the International Conference on Machine Learning (ICML) 2022},
     year = {2022}
   }
   

1. Mehta, Swapneel, Bogdan State, Richard Bonneau, Jonathan Nagler, Philip Torr, and Atılım Güneş Baydin. 2022. “Simulating Social Networks and Disinformation.” In Misinformation Village Co-Hosted by MisinfoCon, DEFCON 30, August 12–13, Las Vegas, NV, USA. https://defcon.misinfocon.com/.
   • Bib

   @inproceedings{mehta-2022-simulation,
     title = {Simulating Social Networks and Disinformation},
     author = {Mehta, Swapneel and State, Bogdan and Bonneau, Richard and Nagler, Jonathan and Torr, Philip and Baydin, Atılım Güneş},
     booktitle = {Misinformation Village co-hosted by {MisinfoCon}, {DEFCON} 30, August 12--13, Las Vegas, NV, USA},
     year = {2022},
     url = {https://defcon.misinfocon.com/}
   }