Timezone: »

Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Ximei Wang · Ying Jin · Mingsheng Long · Jianmin Wang · Michael Jordan

Wed Dec 11 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #46

Deep neural networks (DNNs) excel at learning representations when trained on large-scale datasets. Pre-trained DNNs also show strong transferability when fine-tuned to other labeled datasets. However, such transferability becomes weak when the target dataset is fully unlabeled as in Unsupervised Domain Adaptation (UDA). We envision that the loss of transferability mainly stems from the intrinsic limitation of the architecture design of DNNs. In this paper, we delve into the components of DNN architectures and propose Transferable Normalization (TransNorm) in place of existing normalization techniques. TransNorm is an end-to-end trainable layer to make DNNs more transferable across domains. As a general method, TransNorm can be easily applied to various deep neural networks and domain adaption methods, without introducing any extra hyper-parameters or learnable parameters. Empirical results justify that TransNorm not only improves classification accuracies but also accelerates convergence for mainstream DNN-based domain adaptation methods.

Author Information

Ximei Wang (Tsinghua University)
Ying Jin (Tsinghua University)
Mingsheng Long (Tsinghua University)
Jianmin Wang (Tsinghua University)
Michael Jordan (UC Berkeley)

More from the Same Authors