Poster Thu, Dec 10, 2026 • 10:00 AM – 1:00 PM AEDT Hall 1-4

DataComp-VLM: Improved open datasets for Vision-Language Models

Matteo Farina ⋅ Vishaal Udandarao ⋅ Thao Nguyen ⋅ Selim Kuzucu ⋅ Maximilian Böther ⋅ Andreas Hochlehnert ⋅ Adhiraj Ghosh ⋅ Marianna Nezhurina ⋅ Karsten Roth ⋅ Joschka Strüber ⋅ Yuhui Zhang ⋅ Sebastian Dziadzio ⋅ Elaine Sui ⋅ Soumya S Jahagirdar ⋅ Dhruba Ghosh ⋅ Hasan Hammoud ⋅ Thomas De Min ⋅ Simone Caldarella ⋅ Muhammad Jehanzeb Mirza ⋅ Sedrick Keh ⋅ Mehdi Cherti ⋅ Hilde Kuehne ⋅ Bernt Schiele ⋅ Serena Yeung-Levy ⋅ Muhammad Ferjad Naeem ⋅ Federico Tombari ⋅ Ana Klimovic ⋅ Elisa Ricci ⋅ Matthias Bethge ⋅ Sewoong Oh ⋅ Ameya Prabhu ⋅ Alessio Tonioni ⋅ Jenia Jitsev ⋅ Massimiliano Mancini ⋅ Ludwig Schmidt ⋅ Nikhil Parthasarathy

[ OpenReview]

Abstract

Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we standardize 160 datasets spanning four data types—image-caption pairs, multimodal interleaved documents, text-only, and instruction-tuning data—into a corpus of 6T multimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B–8B models and 6.25B–200B token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream benchmarks across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting dataset, DCVLM-BASELINE, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training tokens. Compared to FINEVISION, the state-of-the-art open VLM training dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available.

Chat is not available.