Skip to yearly menu bar Skip to main content

Workshop: Human in the Loop Learning (HiLL) Workshop at NeurIPS 2022

Time-Efficient Reward Learning via Visually Assisted Cluster Ranking

David Zhang · Micah Carroll · Andreea Bobu · Anca Dragan


One of the most successful paradigms for reward learning uses human feedback in the form of comparisons. Although these methods hold promise, human comparison labeling is expensive and time consuming, constituting a major bottleneck to their broader applicability. Our insight is that we can greatly improve how effectively human time is used in these approaches by batching comparisons together, rather than having the human label each comparison individually. To do so, we leverage data dimensionality-reduction and visualization techniques to provide the human with a interactive GUI displaying the state space, in which the user can label subportions of the state space. Across some simple Mujoco tasks, we show that this high-level approach holds promise and is able to greatly increase the performance of the resulting agents, provided the same amount of human labeling time.

Chat is not available.