$\mathrm{Bounded\mbox{-}RS}$: A Bounded Evaluation Protocol and Testbed for Representational Systematicity
Boyu Cao ⋅ Wenshuo Wang
Abstract
Systematicity is the capacity to generalize compositionally. Existing benchmarks primarily test behavioral systematicity, while representation-level analyses often leave open whether recovered signals are merely decodable or causally used during generation. We therefore introduce $\mathrm{Bounded\mbox{-}RS}$, an evaluation artifact and protocol for adjudicating these possibilities under bounded hypotheses: unsupported, present-only, causal-use, or underdetermined. The artifact is a controlled semantic-parsing setting with matched recombination tests, exact intervention pairs, and evidence routines for recoverability, interchange, necessity, and repair during generation. We apply $\mathrm{Bounded\mbox{-}RS}$ to the two prominent representation-level explanations of systematic generalization, binding and geometry. Within the family instantiations evaluated here, and under the same behavioral anchor and state semantics, the binding family satisfies the criteria for $\mathit{Causal\mbox{-}use}$, whereas the geometry family is adjudicated as $\mathit{Unsupported}$ under its frozen closure basis. Detector-truthfulness tests do not validate a geometry detector, so the geometry conclusion remains a claim about the tested family rather than about all possible geometric mechanisms. $\mathrm{Bounded\mbox{-}RS}$ thus provides a reusable way to make disciplined representational claims from behavioral systematicity tests.
Chat is not available.
Successful Page Load