Timezone: »

A Refined Margin Distribution Analysis for Forest Representation Learning
Shen-Huan Lyu · Liang Yang · Zhi-Hua Zhou

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #6
In this paper, we formulate the forest representation learning approach named casForest as an additive model, and show that the generalization error can be bounded by $\mathcal{O}(\ln m/m)$, when the margin ratio related to the margin standard deviation against the margin mean is sufficiently small. This inspires us to optimize the ratio. To this end, we design a margin distribution reweighting approach for the deep forest model to attain a small margin ratio. Experiments confirm the relation between the margin distribution and generalization performance. We remark that this study offers a novel understanding of casForest from the perspective of the margin theory and further guides the layer-by-layer forest representation learning.

Author Information

Shen-Huan Lyu (Nanjing University)
Liang Yang (Nanjing University)
Zhi-Hua Zhou (Nanjing University)

More from the Same Authors