Same Errors, Different Models: Benchmarking Taxonomic Classification on MassID45
Neelu Madan ⋅ John Quinto ⋅ Scott C. Lowe ⋅ Johanna Orsholm ⋅ Tomas Roslin ⋅ Tommi Mononen ⋅ Nicolas Chazot ⋅ Brendan Furneaux ⋅ Otso Ovaskainen ⋅ Andreas Møgelmose ⋅ Thomas Moeslund ⋅ Graham Taylor
Abstract
Automated taxonomic classification from bulk-trap insect imagery remains challenging due to extreme size and aspect-ratio variation and severe class imbalance. We introduce a taxonomic classification benchmark, MassID45-Cls, built from the MassID45 dataset. The original dataset contains expert-verified segmentations and fine-grained labels, but no classification benchmark. We bridge this gap by cropping individual specimens directly from the bulk imagery to construct a formal benchmark of taxonomically annotated images. Images are labelled at the highest taxonomic resolution identifiable from a photograph alone, with residual ``catch-all'' labels where finer resolution was not possible. This results in 15,023 labelled specimens across 34 classes, with specimen crops ranging from mites of roughly 8$\times$10 px to bees of roughly 171$\times$206 px, and severe long-tailed class frequency. We benchmark six different models and find that all converge to a similar performance ceiling ($\sim$70--74\% Top-1), with per-class errors highly correlated across architecturally distinct models (Spearman's $\rho = 0.98$-$0.99$). This strong correlation suggests that classification difficulty is an intrinsic property of the data. However, our setup cannot rule out inductive biases across all six architectures. For example, patch- or grid-based processing, which may struggle with irregular, non-square specimens regardless of resolution strategy. We further show that roughly 60\% of fine-grained errors remain within the correct taxonomic order rather than crossing order boundaries, with the largest remaining source of error concentrated in catch-all labels (e.g., Diptera (Other), 14.8\% recall) that are defined by exclusion rather than shared morphology.
Chat is not available.
Successful Page Load