Timezone: »

 
Poster
AutoMS: Automatic Model Selection for Novelty Detection with Error Rate Control
Yifan Zhang · Haiyan Jiang · Haojie Ren · Changliang Zou · Dejing Dou

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #235
in Poster Session 2 »

Given an unsupervised novelty detection task on a new dataset, how can we automatically select a ''best'' detection model while simultaneously controlling the error rate of the best model? For novelty detection analysis, numerous detectors have been proposed to detect outliers on a new unseen dataset based on a score function trained on available clean data. However, due to the absence of labeled data for model evaluation and comparison, there is a lack of systematic approaches that are able to select a ''best'' model/detector (i.e., the algorithm as well as its hyperparameters) and achieve certain error rate control simultaneously. In this paper, we introduce a unified data-driven procedure to address this issue. The key idea is to maximize the number of detected outliers while controlling the false discovery rate (FDR) with the help of Jackknife prediction. We establish non-asymptotic bounds for the false discovery proportions and show that the proposed procedure yields valid FDR control under some mild conditions. Numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the effectiveness of our proposed AutoMS method. The code is available at https://github.com/ZhangYifan1996/AutoMS.

Keywords: model selection · FDR · Novelty Detection
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Yifan Zhang (Nankai University)
Yifan Zhang

Zhang Yifan is a Ph.D. student in Statistics, from School of Statistics and Data Science, Nankai University, China. Her Ph.D. supervisor is [Prof. Changliang Zou](http://web.stat.nankai.edu.cn/chlzou/). Her research interests include change-point analysis, outlier detection, and statistical process control. She was awarded university scholarships in 2014, 2015, 2017, 2019, 2021. If you are interested in her research topics, please feel free to contact her via email: yfzhang_stat@mail.nankai.edu.cn.

Haiyan Jiang (MBZUAI)
Haojie Ren (Shanghai Jiaotong University)
Changliang Zou
Dejing Dou (Baidu)

More from the Same Authors