EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards

Earth-system investigations reconstruct changing physical processes from observations that differ in scale, timing, variable definition, and modality. EarthVerse evaluates whether scientific agents can carry out this work across a complete natural-hazard investigation. Its 405 tasks cover 199 documented events, 19 hazard families, 10,879 scored answer units, and more than 80 data-product and access families. Agents must find compatible evidence, calculate, reconcile sources, and preserve support for the final account. We evaluate 25 systems under a controlled tool-using protocol. The best mean answer-unit accuracy is 84.65%, but the highest Strict@95 is only 34.81%. Additional reasoning helps when interaction can revise the current claim--evidence state and often hurts when evidence is fixed. Revealing the relevant files raises Core by 14.72 points, far more than a planning oracle or a curated tool subset. These results locate the main difficulty before and between calculations: an agent must keep evidence, derived values, and physical interpretation consistent as the investigation changes.