A Cooperation Index for Model Pruning

In complex models, pruning redundant parameters reveals its core functional elements, improving both generalizability and interpretability. Tools for effective pruning rely on criteria to identify unnecessary parameters, often serving as decision-making agents by analyzing each parameter’s contribution in combination with others. The Shapley Value (SV) has recently been considered such a criterion, which we interpret as measuring the marginal contribution across all possible paths of parameter accumulation. However, we find that SV systematically overweights redundant parameters due to its averaging process. Instead, measuring the speed of decay of the marginal contribution when parameters are combined can serve as a more effective decision criterion. We quantify the number of cooperative contribution of parameters and show that using this criterion is more effective for parameter pruning in backward elimination, leading to a more optimal set of remaining parameters.