Reliable Anatomical Landmarking of Histological Serial Sections with Reasoning Agents
Abstract
Spatial association between markers in single-plex immunohistochemistry requires consecutive sections to be brought into alignment, and the requisite landmarks are obtained by manual annotation. Automation of this task uses a dense feature matcher to provide the correspondences and a Fitzpatrick-West model to predict the target registration error rather than measuring it against a second annotator. However, variance in tissue anatomy and the rotation between sections have made this automation hard. The main hypothesis of this paper is that reasoning agents can identify the rotation when asked and annotate landmarks independent of the transform while accounting for anatomical variance, rejecting or placing landmarks and flagging uncertainties for human review. The results show that on whole slide images reasoning agents recover the rotation to within 1.3 degrees on all 28 ANHIR pairs evaluated, with a median error of 0.26 degrees, and place landmarks with an error on par with that of a second annotator. For field crops, the agents come within 11 µm of the expert landmarks where those landmarks are accurate, in addition to recovering the rotation to within 2.9 degrees on all 8 MIIT pairs evaluated. While a deterministic search over angles recovers the rotation on most pairs, in the cases where it fails it does so silently and returns a false rotation confidently. The matcher's error is set by the angle between the sections rather than by the tissue or the stain, while the Fitzpatrick-West model that would stand in for the second annotator carries no term for tissue deformation, which is what makes it optimistic. This paper highlights that reasoning agents can act in place of expert annotators while producing annotations that can be relied on, since their reasoning lets them adjudicate landmark candidates on anatomy and flag the ones where no candidate can be found, allowing single-plex spatial association to stand as a viable alternative to multiplex imaging.