Beyond IPS: Reliable Counterfactual Evaluation in Multi-Stage Ad Systems without Logged Propensities

Offline evaluation of ad ranking policies in multi stage delivery systems is fundamentally challenging because action propensities are often unavailable, limiting inverse propensity scoring (IPS) estimators. While reward model based estimators such as Direct Method (DM) do not rely on propensity estimates, they can suffer from bias when candidate policies induce impression distributions that differ from the logging mixture. In this work, we formalize DM based counterfactual replay under mixture logging and propose a mixture targeted shift correction based on density ratio weighting through domain classifier. We further derive a per policy value estimation error and characterize an asymptotic error ceiling governed by the cold start mass. Experiments on both ads delivery simulator and one week of production traffic from a large scale industrial CTR ranking system demonstrate that the proposed mixture weighted reward model consistently outperforms its unweighted counterpart under distribution shift.