Timezone: »

 
Poster
CatBoost: unbiased boosting with categorical features
Liudmila Prokhorenkova · Gleb Gusev · Aleksandr Vorobev · Anna Veronika Dorogush · Andrey Gulin

Thu Dec 06 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #114
in Thu Poster Session B »

This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.

Keywords: Boosting and Ensemble Methods

Author Information

Liudmila Prokhorenkova (Yandex)
Gleb Gusev (Yandex LLC)
Aleksandr Vorobev (Yandex LLC)
Anna Veronika Dorogush (Yandex)
Andrey Gulin (Yandex)

More from the Same Authors