The Operating Point Moves More Than the Ranking
Anqi Peter Li
Abstract
Can monitor-aware meta-agent search move a monitor's operating point without moving its ranking? The monitor is a linear probe on residual-stream activations, scoring matched harmful and benign requests. In a synthetic finite-harness pilot on frozen open-weight workers we read both endpoints on one contrast, across 12 harvested v6 black-box runs: aware against monitor-blind search, same task objective, never reading the monitor, so the answer does not turn on units. The frozen-threshold recall loss is $+0.0367$ $[+0.0113,+0.0620]$, excluding zero; report AUROC changes by $-0.0011$ $[-0.0064,+0.0042]$, covering it. Reading the monitor relocates the operating point without reordering it. Effective $n$ is 10: two seeds' arms converge. Relative to baseline, the aware search changes recall at the frozen threshold by $-0.0533$ $[-0.0929,-0.0138]$ and the false-positive rate by $-0.0600$ $[-0.0983,-0.0217]$, same-sign changes resolved only to $0.04$ per seed, while monitor-blind design search shows no detectable change in either endpoint ($-0.0167$ $[-0.0564,+0.0231]$ TPR, $+0.0033$ $[-0.0656,+0.0723]$ FPR). Against its own baseline the aware arm's report AUROC changes by $-0.0051$ $[-0.0092,-0.0010]$, an interval excluding zero but equivalent to 3.2 of 625 report-set ranking steps, and the arms that never read the monitor move it about as far (blind and random arms: $-0.0040$, $-0.0029$), so a before/after reading credits awareness with a change ignoring the monitor also produces. Three qualifications: the shift is near this design's detection limit; refitting the threshold on attacked calibration data returns recall to baseline within noise ($+0.0133$ $[-0.0160,+0.0427]$, an interval that includes zero and excludes the frozen-threshold loss); and the preserved ranking is partly induced by an aware objective that is a pooled panel mean and never rewards reordering. We read this as monitor maintenance debt on a one-search-run timescale, not monitor blindness; the topic-disjoint probe shows no detectable degradation ($-0.0044$ $[-0.0131,+0.0042]$). A 16-record capability-constrained gradient sensitivity, whose objective is not a panel mean, does alter ranking, though pooled over a pre-specified and a post-hoc step size: pooled report AUROC drop $+0.1449$ $[+0.0523,+0.2375]$; the pre-specified arm alone spans zero.
Chat is not available.
Successful Page Load