Timezone: »
Thanks to the powerful representation capabilities, transformers have made impressive progress in image restoration. However, existing transformers-based methods do not carefully consider the particularities of image restoration. In general, image restoration requires that an ideal approach should be translation-invariant to the degradation, i.e., the undesirable degradation should be removed irrespective of its position within the image. Furthermore, the local relationships also play a vital role, which should be faithfully exploited for recovering clean images. Nevertheless, most transformers either adopt local attention with the fixed local window strategy or global attention, which unfortunately breaks the translation invariance and causes huge loss of local relationships. To address these issues, we propose an elegant stochastic window strategy for transformers. Specifically, we first introduce the window partition with stochastic shift to replace the original fixed window partition for training. Then, we design a new layer expectation propagation algorithm to efficiently approximate the expectation of the induced stochastic transformer for testing. Our stochastic window transformer not only enjoys powerful representation but also maintains the desired property of translation invariance and locality. Experiments validate the stochastic window strategy consistently improves performance on various image restoration tasks (deraining, denoising and deblurring) by significant margins. The code is available at https://github.com/jiexiaou/Stoformer.
Author Information
Jie Xiao (University of Science and Technology of China)
Xueyang Fu (University of Science and Technology of China)
Feng Wu
Zheng-Jun Zha (University of Science and Technology of China)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: Stochastic Window Transformer for Image Restoration »
Dates n/a. Room
More from the Same Authors
-
2022 Poster: Exploring Figure-Ground Assignment Mechanism in Perceptual Organization »
Wei Zhai · Yang Cao · Jing Zhang · Zheng-Jun Zha -
2022 Poster: Rank Diminishing in Deep Neural Networks »
Ruili Feng · Kecheng Zheng · Yukun Huang · Deli Zhao · Michael Jordan · Zheng-Jun Zha -
2023 Poster: DreamWaltz: Make a Scene with Complex 3D Animatable Avatars »
Yukun Huang · Jianan Wang · Ailing Zeng · He CAO · Xianbiao Qi · Yukai Shi · Zheng-Jun Zha · Lei Zhang -
2023 Poster: State Sequences Prediction via Fourier Transform for Representation Learning »
Mingxuan Ye · Yufei Kuang · Jie Wang · Yang Rui · Wengang Zhou · Houqiang Li · Feng Wu -
2023 Poster: DAW: Exploring the Better Weighting Function for Semi-supervised Semantic Segmentation »
Rui Sun · Huayu Mai · Tianzhu Zhang · Feng Wu -
2023 Poster: A Deep Instance Generative Framework For MILP Solvers Under Limited Data Availability »
Zijie Geng · Jie Wang · Xijun Li · Xiao Li · Yongdong Zhang · Feng Wu -
2022 Spotlight: Lightning Talks 5B-3 »
Yanze Wu · Jie Xiao · Nianzu Yang · Jieyi Bi · Jian Yao · Yiting Chen · Qizhou Wang · Yangru Huang · Yongqiang Chen · Peixi Peng · Yuxin Hong · Xintao Wang · Feng Liu · Yining Ma · Qibing Ren · Xueyang Fu · Yonggang Zhang · Kaipeng Zeng · Jiahai Wang · GEN LI · Yonggang Zhang · Qitian Wu · Yifan Zhao · Chiyu Wang · Junchi Yan · Feng Wu · Yatao Bian · Xiaosong Jia · Ying Shan · Zhiguang Cao · Zheng-Jun Zha · Guangyao Chen · Tianjun Xiao · Han Yang · Jing Zhang · Jinbiao Chen · MA Kaili · Yonghong Tian · Junchi Yan · Chen Gong · Tong He · Binghui Xie · Yuan Sun · Francesco Locatello · Tongliang Liu · Yeow Meng Chee · David P Wipf · Tongliang Liu · Bo Han · Bo Han · Yanwei Fu · James Cheng · Zheng Zhang -
2021 Poster: Low-Rank Subspaces in GANs »
Jiapeng Zhu · Ruili Feng · Yujun Shen · Deli Zhao · Zheng-Jun Zha · Jingren Zhou · Qifeng Chen -
2021 Poster: ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs »
Zhanqiu Zhang · Jie Wang · Jiajun Chen · Shuiwang Ji · Feng Wu -
2021 Poster: Unfolding Taylor's Approximations for Image Restoration »
man zhou · Xueyang Fu · Zeyu Xiao · Gang Yang · Aiping Liu · Zhiwei Xiong -
2020 Poster: Hierarchical Granularity Transfer Learning »
Shaobo Min · Hongtao Xie · Hantao Yao · Xuran Deng · Zheng-Jun Zha · Yongdong Zhang -
2020 Poster: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2020 Spotlight: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2019 Poster: Learning Deep Bilinear Transformation for Fine-grained Image Representation »
Heliang Zheng · Jianlong Fu · Zheng-Jun Zha · Jiebo Luo -
2019 Poster: Abstract Reasoning with Distracting Features »
Kecheng Zheng · Zheng-Jun Zha · Wei Wei