Adaptivee: Adaptive Ensemble for Tabular Data

Ensemble methods are widely used to improve model performance by combining multiple models, each contributing uniquely to predictions. Traditional ensemble approaches often rely on static weighting schemes that do not account for the varying effectiveness of individual models across different subspaces of the data. This work introduces adaptivee, a dynamic ensemble framework designed to optimize performance for tabular data tasks by adjusting model weights in response to specific data characteristics. The adaptivee framework offers flexibility through various reweighting strategies, including emphasizing single models for subspace specialization or distributing importance among models for robustness. Experiments on the OpenML-CC18 benchmark demonstrate that adaptivee can significantly boost performance, achieving up to a 0.6\% improvement in balanced accuracy over AutoGluon ensembling strategies. This framework opens new avenues for advancing ensemble techniques, particularly in tabular data contexts where model complexity is constrained by the nature of the data.