MEWS: Real-time Social Media Manipulation Detection and Analysis

One of the most challenging aspects of online disinformation is the overwhelming volume of content that is published on social media platforms. For example, hundreds of thousands of images and videos are uploaded to Facebook every minute. Organizing and analyzing this volume of content in the hope of detecting disinformation campaigns in (near) real-time is impossible for humans without the assistance of automated AI tools. This problem is especially pertinent in young and struggling democracies whose traditional media organizations lack the ability to keep pace with the explosion of deep-fake, manipulated, altered or plainly-fake online media. In an effort to provide such capacity, we have developed a real-time social media manipulation detection and analysis system called MEWS (Misinformation Early Warning System).

This system combines work in digital forensics, computer vision, graph analysis, and media studies to accomplish three specific tasks: (1) MEWS ingests enormous amounts of images and video from various social media platforms (e.g., Facebook, Instagram, Twitter, Telegram) using keyword targets provided by partner media organizations from across the world; (2) MEWS employs state-of-the-art AI systems to detect and extract faces, objects, text (including meme-text), image features, and any potential manipulations from the visual content; and (3) MEWS constructs a media-graph which pairs similar sub-images, objects, and manipulations for display in an interactive, easily-navigable, and searchable user interface.

We offer a demonstration of MEWS' organizational and analytic capabilities using tens of millions of images (and other media) collected from several social media platforms (Facebook, Instagram, and Twitter) in the Indonesian context.