Timezone: »

Unbiased Efficient Feature Counts for Inverse RL
Gerard Donahue · Brendan Crowe · Marek Petrik · Daniel Brown

Feature counts play a crucial role when computing good reward weights in inverse reinforcement learning. Despite their importance, little work has focused on developing better methods for estimating feature counts. In this work, we propose a new method for estimating feature counts for scenarios with a small number of long demonstrations. Most existing algorithms perform poorly in this scenario. In particular, we propose two new algorithms, E-DLS and E-SLS, which can efficiently use a small number of long demonstrations to estimate feature counts. We show that E-SLS estimates are unbiased, which is the first such estimation algorithm. Our experimental results on benchmark problems demonstrate better learned reward weights when feature counts are estimated with E-DLS and E-SLS compared to other popular methods.

Author Information

Gerard Donahue (University of New Hampshire)
Brendan Crowe (UNH)
Marek Petrik (University of New Hampshire)
Daniel Brown (UC Berkeley)

More from the Same Authors