Cynthia Rudin: The Marriage of Noise and Simplicity

Noisier data generally leads to simpler machine learning models. My collaborators and I have been studying this phenomenon over many years, aiming to understand why tabular datasets (which often have inherent noise) often admit simple-yet-accurate models. Our latest work provides analytical relationships between noise and implicit sparsity regularization, showing that more noise leads to more regularization (and thus simpler models). These results are exciting because they help explain why more adding more model complexity doesn’t always work, and why, in many cases, it simply cannot work. Policy makers can use this information to understand what levels of complexity are truly needed for certain types of high-stakes decisions.

Zachery Boner*, Lesia Semenova*, Harry Chen*, Cynthia Rudin, Ronald Parr Using Noise to Infer Aspects of Simplicity Without Learning. NeurIPS, 2024. https://openreview.net/pdf?id=b172ac0R4L