Timezone: »

 
Spotlight
Stochastic Window Transformer for Image Restoration
Jie Xiao · Xueyang Fu · Feng Wu · Zheng-Jun Zha

Thu Dec 08 09:00 AM -- 11:00 AM (PST) @
in Lightning Talks 5B-3 »

Thanks to the powerful representation capabilities, transformers have made impressive progress in image restoration. However, existing transformers-based methods do not carefully consider the particularities of image restoration. In general, image restoration requires that an ideal approach should be translation-invariant to the degradation, i.e., the undesirable degradation should be removed irrespective of its position within the image. Furthermore, the local relationships also play a vital role, which should be faithfully exploited for recovering clean images. Nevertheless, most transformers either adopt local attention with the fixed local window strategy or global attention, which unfortunately breaks the translation invariance and causes huge loss of local relationships. To address these issues, we propose an elegant stochastic window strategy for transformers. Specifically, we first introduce the window partition with stochastic shift to replace the original fixed window partition for training. Then, we design a new layer expectation propagation algorithm to efficiently approximate the expectation of the induced stochastic transformer for testing. Our stochastic window transformer not only enjoys powerful representation but also maintains the desired property of translation invariance and locality. Experiments validate the stochastic window strategy consistently improves performance on various image restoration tasks (deraining, denoising and deblurring) by significant margins. The code is available at https://github.com/jiexiaou/Stoformer.

Keywords: transformer · Image restoration · translation invariance · stochastic window strategy · locality

Author Information

Jie Xiao (University of Science and Technology of China)
Xueyang Fu (University of Science and Technology of China)
Feng Wu
Zheng-Jun Zha (University of Science and Technology of China)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors