MAMORX: Multi-agent Multi-Modal Scientific Review Generation with External Knowledge

The deluge of scientific papers has made it challenging for researchers to thoroughly engage with the expanding literature. We propose MAMORX, a new automated scientific review generation system that relies on multi-modal foundation models to address this challenge. MAMORX replicates key aspects of human review by integrating attention to text, figures, and citations, along with access to external knowledge sources. Compared to previous work, it takes advantage of large context windows to significantly reduce the number of agents and the processing time needed. The system relies on structured outputs and function calling to process figures, evaluate novelty, and build general and domain-specific knowledge bases from external scholarly search. To test our system, we conducted an arena-style competition between several baselines and human reviews on diverse papers from general machine learning and NLP fields, calculating an Elo rating for each model based on human preferences. MAMORX has an estimated win rate of 93% against human reviews and outperforms the next-best model, a multi-agent baseline system, losing only 12% of the time and never losing against other prominent models. We share our system (https://anonymous.4open.science/r/MAMORX-BD44), and discuss further applications of foundation models, especially multi-modal ones, for scientific evaluation.