Skip to yearly menu bar Skip to main content


Poster

Addressing Hidden Confounding with Heterogeneous Observational Datasets for Recommendation

Yanghao Xiao · Haoxuan Li · Yongqiang Tang · Wensheng Zhang


Abstract:

The collected data in recommender systems generally suffers selection bias. Considerable works are proposed to address selection bias induced by observed user and item features, but they fail when hidden features (e.g., user age or salary) that affect both user selection mechanism and feedback exist, which is called hidden confounding. To tackle this issue, methods based on sensitivity analysis and leveraging a few RCT data for model calibration have been proposed. However, the former relies on strong assumptions of hidden confounding strength, whereas the latter relies on the expensive RCT data, thereby limiting their applicability in real-world scenarios. In this paper, we propose to employ heterogeneous observational data to address hidden confounding, wherein some data is subject to hidden confounding while the remaining is not. We argue that such setup is more aligned with practical scenarios, especially when some users do not have complete personal information (thus assumed with hidden confounding), while others do have (thus assumed without hidden confounding). To achieve unbiased learning, we propose a novel meta-learning based debiasing method called MetaDebias, to explicitly model the oracle prediction errors and the additional bias introduced by hidden confounding, and bi-level optimization is used for model training. Extensive experiments on three public datasets validate our method achieves state-of-the-art performance in the presence of hidden confounding, regardless of RCT data availability.

Live content is unavailable. Log in and register to view live content