Reference Representation Affects Quantity Preferences in Protocol Evaluation
Abstract
Protocol evaluators compare generated laboratory instructions with a reference, whose unit representation can affect judgments of quantity agreement. On 21 protocol records, converting only the selected reference quantity to an equivalent unit makes an unchanged candidate score below its doubled-quantity variant in 15 of 21 SciBERT comparisons. We diagnose this dependence using five controlled candidate conditions, eight BioPlanner reconstruction components, and ten BioProBench generation components. With the original reference, no component prefers equivalent unit conversion to doubling in the original unit. Argument SciBERT and all five GEN lexical scores prefer doubling in all 21 records. Replacing selected micro-sign units with ASCII or Greek spellings removes their unknown tokens while preserving the primary SciBERT reversals. Canonicalizing references and candidates separately makes quantity-faithful bodies reference-identical, providing a preprocessing positive control. These constructed cases do not measure natural error prevalence, biological consequences, or agent-selection benefit.