Timezone: »

 
Poster
Feedback Detection for Live Predictors
Stefan Wager · Nick Chamandy · Omkar Muralidharan · Amir Najmi

Tue Dec 09 04:00 PM -- 08:59 PM (PST) @ Level 2, room 210D
in Poster Session »

A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze predictor feedback detection as a causal inference problem, and introduce a local randomization scheme that can be used to detect non-linear feedback in real-world problems. We conduct a pilot study for our proposed methodology using a predictive system currently deployed as a part of a search engine.

[ PDF] 

Author Information

Stefan Wager (Stanford University)
Nick Chamandy (Google)
Omkar Muralidharan (Google)
Amir Najmi (Google)

More from the Same Authors