Timezone: »
In stochastic optimal control the distribution of the exogenous noise is typically unknown and must be inferred from limited data before dynamic programming (DP)-based solution schemes can be applied. If the conditional expectations in the DP recursions are estimated via kernel regression, however, the historical sample paths enter the solution procedure directly as they determine the evaluation points of the cost-to-go functions. The resulting data-driven DP scheme is asymptotically consistent and admits efficient computational solution when combined with parametric value function approximations. If training data is sparse, however, the estimated cost-to-go functions display a high variability and an optimistic bias, while the corresponding control policies perform poorly in out-of-sample tests. To mitigate these small sample effects, we propose a robust data-driven DP scheme, which replaces the expectations in the DP recursions with worst-case expectations over a set of distributions close to the best estimate. We show that the arising min-max problems in the DP recursions reduce to tractable conic programs. We also demonstrate that this robust algorithm dominates state-of-the-art benchmark algorithms in out-of-sample tests across several application domains.
Author Information
Grani Adiwena Hanasusanto (Imperial College London)
Daniel Kuhn (EPFL)
More from the Same Authors
-
2021 Poster: Robust Generalization despite Distribution Shift via Minimum Discriminating Information »
Tobias Sutter · Andreas Krause · Daniel Kuhn -
2020 : Invited Talk 4: From Moderate Deviations Theory to Distributionally Robust Optimization: Learning from Correlated Data »
Daniel Kuhn -
2019 : Daniel Kuhn »
Daniel Kuhn -
2019 : Daniel Kuhn: From Data to Decisions: Distributionally Robust Optimization is Optimal »
Daniel Kuhn -
2019 Poster: Calculating Optimistic Likelihoods Using (Geodesically) Convex Optimization »
Viet Anh Nguyen · Soroosh Shafieezadeh Abadeh · Man-Chung Yue · Daniel Kuhn · Wolfram Wiesemann -
2019 Poster: Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation »
Viet Anh Nguyen · Soroosh Shafieezadeh Abadeh · Man-Chung Yue · Daniel Kuhn · Wolfram Wiesemann -
2018 Poster: Wasserstein Distributionally Robust Kalman Filtering »
Soroosh Shafieezadeh Abadeh · Viet Anh Nguyen · Daniel Kuhn · Peyman Mohajerin Esfahani -
2018 Spotlight: Wasserstein Distributionally Robust Kalman Filtering »
Soroosh Shafieezadeh Abadeh · Viet Anh Nguyen · Daniel Kuhn · Peyman Mohajerin Esfahani -
2015 Poster: Distributionally Robust Logistic Regression »
Soroosh Shafieezadeh Abadeh · Peyman Esfahani · Daniel Kuhn -
2015 Spotlight: Distributionally Robust Logistic Regression »
Soroosh Shafieezadeh Abadeh · Peyman Esfahani · Daniel Kuhn