Timezone: »

SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning
Sungmin Cha · beomyoung kim · YoungJoon Yoo · Taesup Moon

Tue Dec 07 04:30 PM -- 06:00 PM (PST) @

We consider a class-incremental semantic segmentation (CISS) problem. While some recently proposed algorithms utilized variants of knowledge distillation (KD) technique to tackle the problem, they only partially addressed the key additional challenges in CISS that causes the catastrophic forgetting; \textit{i.e.}, the semantic drift of the background class and multi-label prediction issue. To better address these challenges, we propose a new method, dubbed as SSUL-M (Semantic Segmentation with Unknown Label with Memory), by carefully combining several techniques tailored for semantic segmentation. More specifically, we make three main contributions; (1) modeling \textit{unknown} class within the background class to help learning future classes (help plasticity), (2) \textit{freezing} backbone network and past classifiers with binary cross-entropy loss and pseudo-labeling to overcome catastrophic forgetting (help stability), and (3) utilizing \textit{tiny exemplar memory} for the first time in CISS to improve \textit{both} plasticity and stability. As a result, we show our method achieves significantly better performance than the recent state-of-the-art baselines on the standard benchmark datasets. Furthermore, we justify our contributions with thorough and extensive ablation analyses and discuss different natures of the CISS problem compared to the standard class-incremental learning for classification. The official code is available at https://github.com/clovaai/SSUL.

Author Information

Sungmin Cha (Seoul National University)
beomyoung kim (NAVER corp)
YoungJoon Yoo (Seoul National University)
Taesup Moon (Seoul National University (SNU))

Taesup Moon is currently an associate professor at Seoul National University (SNU), Korea. Prior to joining SNU in 2021, he was an associate professor at Sungkyunkwan University (SKKU) from 2017 to 2021, an assistant professor at DGIST from 2015 to 2017, a research staff member at Samsung Advanced Institute of Technology (SAIT) from 2013 to 2015, a postdoctoral researcher at UC Berkeley, Statistics from 2012 to 2013, and a research scientist at Yahoo! Labs from 2008 to 2012. He got his Ph.D. and MS degrees in Electrical Engineering from Stanford University, CA USA in 2008 and 2004, respectively, and his BS degree in Electrical Engineering from Seoul National University, Korea in 2002. His research interests are in deep learning, statistical machine learning, data science, signal processing, and information theory.

More from the Same Authors