Timezone: »
Poster
Faster Boosting with Smaller Memory
Julaiti Alafate · Yoav S Freund
Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #7
State-of-the-art implementations of boosting, such as XGBoost and LightGBM, can process large training sets extremely fast. However, this performance requires that the memory size is sufficient to hold a 2-3 multiple of the training set size. This paper presents an alternative approach to implementing the boosted trees, which achieves a significant speedup over XGBoost and LightGBM, especially when the memory size is small. This is achieved using a combination of three techniques: early stopping, effective sample size, and stratified sampling. Our experiments demonstrate a 10-100 speedup over XGBoost when the training data is too large to fit in memory.
Author Information
Julaiti Alafate (University of California San Diego)
Yoav S Freund (University of California, San Diego)
More from the Same Authors
-
2022 Poster: When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint »
Yoav S Freund · Yi-An Ma · Tong Zhang -
2016 Poster: Optimal Binary Classifier Aggregation for General Losses »
Akshay Balsubramani · Yoav S Freund -
2007 Demonstration: Automatic Cameraman »
Yoav S Freund · Evan Ettinger · Brian McFee · Deborah Goshorn · Shankar Shivappa -
2007 Poster: Learning the structure of manifolds using random projections »
Yoav S Freund · Sanjoy Dasgupta · Mayank Kabra · Nakul Verma