PACE: Diagnosing Action Aliasing in Concept-Bottleneck Policies Before Retraining

A concept-bottleneck policy is constrained by a fixed concept set. When a decision depends on information that the concepts do not carry, missing information can lead to a policy failure. We introduce PACE (Partition-Aliasing Concept Evaluation), which uses a fixed teacher policy and asks whether adding a candidate concept makes its actions easier to predict from the bottleneck. PACE also shows where the missing information matters. In Craftax-Classic, PACE identifies missing sapling inventory for place_plant; adding this single concept and retraining substantially improves task success. We also establish a limitation for PACE: because it is computed from states visited by the teacher, its interpretation for a failing policy requires that policy to reach the relevant states, and even then a positive diagnosis does not guarantee repair. PACE therefore diagnoses and localizes missing information in concept-bottleneck policies rather than predicting whether retraining will repair a policy failure.