Adaptive Data Analysis
Vitaly Feldman · Aaditya Ramdas · Aaron Roth · Adam Smith

Fri Dec 9th 08:00 AM -- 06:30 PM @ Room 122 + 123
Event URL: »

Adaptive data analysis is the increasingly common practice by which insights gathered from data are used to inform further analysis of the same data sets. This is common practice both in machine learning, and in scientific research, in which data-sets are shared and re-used across multiple studies. Unfortunately, most of the statistical inference theory used in empirical sciences to control false discovery rates, and in machine learning to avoid overfitting, assumes a fixed class of hypotheses to test, or family of functions to optimize over, selected independently of the data. If the set of analyses run is itself a function of the data, much of this theory becomes invalid, and indeed, has been blamed as one of the causes of the crisis of reproducibility in empirical science.

Recently, there have been several exciting proposals for how to avoid overfitting and guarantee statistical validity even in general adaptive data analysis settings. The problem is important, and ripe for further advances. The goal of this workshop is to bring together members of different communities (from machine learning, statistics, and theoretical computer science) interested in solving this problem, to share recent results, to discuss promising directions for future research, and to foster collaborations.

08:55 AM Introductory remarks (Introduction)
09:00 AM Ruth Heller. Inference following aggregate level hypothesis testing in large scale genomic data (Talk)
09:35 AM Weijie Su. Private false discovery rate control and robustness of the Benjamini-Hochberg procedure (Talk)
10:10 AM Vitaly Feldman (Discussion) Vitaly Feldman
10:20 AM Coffee break (break)
10:50 AM Short talks: Ibrahim Alabdulmohsin, Joshua Loftus, Yu-Xiang Wang, Sam Elder, Aaditya Ramdas, Ryan Rogers (Talk)
12:00 PM Lunch break (break)
02:30 PM Aaron Roth. Adaptive Data Analysis via Differential Privacy (Talk)
03:05 PM Katrina Ligett. Adaptive Learning with Robust Generalization Guarantees (Talk)
03:50 PM Posters (Poster Session)
04:35 PM Lucas Janson. Model-free knockoffs: statistical tools for reproducible selections (Talk)
04:55 PM Xiaoying Harris. From Selective Inference to Adaptive Data Analysis (Talk)
05:15 PM Peter Grunwald. Safe Testing: An Adaptive Alternative to p-value-based testing (Talk)
05:50 PM Aaron Roth (Discussion)

Author Information

Vitaly Feldman (Google Brain)
Aaditya Ramdas (UC Berkeley)
Aaron Roth (University of Pennsylvania)
Adam Smith (Pennsylvania State University)

More from the Same Authors