Timezone: »

 
Bandits for Online Calibration: An Application to Content Moderation on Social Media Platforms
Vashist Avadhanula · Omar Abdul Baki · Hamsa Bastani · Osbert Bastani · Caner Gocmen · Daniel Haimovich · Darren Hwang · Dmytro Karamshuk · Thomas Leeper · Jiayuan Ma · Gregory macnamara · Jake Mullet · Christopher Palow · Sung Park · Varun S Rajagopal · Kevin Schaeffer · Parikshit Shah · Deeksha Sinha · Nicolas Stier-Moses · Ben Xu

@
in Challenges in Deploying and Monitoring Machine Learning Systems »

We describe the current content moderation strategy employed by Meta to remove policy-violating content from its platforms. Meta relies on both handcrafted and learned risk models to flag potentially violating content for human review. Our approach aggregates these risk models into a single ranking score, calibrating them to prioritize more reliable risk models. A key challenge is that violation trends change over time, affecting which risk models are most reliable. Our system additionally handles production challenges such as changing risk models and novel risk models. We use a contextual bandit to update the calibration in response to such trends. Our approach increases Meta's top-line metric for measuring the effectiveness of its content moderation strategy by 13%.

Author Information

Vashist Avadhanula (Facebook)

Vashist is a research scientist on the Core Data Science team. At Facebook, he works on two broad themes: (1) leveraging queueing theory and optimization to streamline Facebook's content moderation framework and (2) helping various product teams implement state of the art multi-arm bandits/reinforcement learning algorithms to improve key product metrics. He obtained his PhD in Decision, Risk and Operations from Columbia University, where he worked on developing tractable online learning algorithms for choice models. As part of his thesis work, he has collaborated with Flipkart, India's largest e-commerce firm in improving their product recommendations. He is also an active reviewer/program committee member for a number of AI/ML conferences including ICML, NeurIPS, AISTATS.

Omar Abdul Baki (Meta)
Hamsa Bastani (Wharton School, University of Pennsylvania)

My research focuses on developing novel machine learning algorithms for data-driven decision-making, with applications to healthcare operations, revenue management, and social good. Recently, I've been working on the design and application of transfer learning algorithms, e.g., for predictive analytics with small data, dynamic pricing across related products, and speeding up clinical trials with surrogate outcomes. I am also interested in algorithmic accountability and using big data to combat social and environmental harm.

Osbert Bastani (University of Pennsylvania)
Caner Gocmen (Facebook)
Daniel Haimovich (Facebook)
Darren Hwang (Meta)
Dmytro Karamshuk (Facebook)
Thomas Leeper (Meta)
Jiayuan Ma (Meta)
Gregory macnamara (Meta)
Jake Mullet (Meta)
Christopher Palow (Meta)
Sung Park (Meta)
Varun S Rajagopal (Meta)
Kevin Schaeffer (Facebook)
Parikshit Shah (Facebook)
Deeksha Sinha (Meta)
Nicolas Stier-Moses (Meta)
Ben Xu (Meta)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors