Timezone: »

 
Poster
Swapping Autoencoder for Deep Image Manipulation
Taesung Park · Jun-Yan Zhu · Oliver Wang · Jingwan Lu · Eli Shechtman · Alexei Efros · Richard Zhang

Mon Dec 07 09:00 PM -- 11:00 PM (PST) @ Poster Session 0 #105
in Poster Session 1 »

Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging. We propose the Swapping Autoencoder, a deep model designed specifically for image manipulation, rather than random sampling. The key idea is to encode an image into two independent components and enforce that any swapped combination maps to a realistic image. In particular, we encourage the components to represent structure and texture, by enforcing one component to encode co-occurrent patch statistics across different parts of the image. As our method is trained with an encoder, finding the latent codes for a new input image becomes trivial, rather than cumbersome. As a result, our method enables us to manipulate real input images in various ways, including texture swapping, local and global editing, and latent code vector arithmetic. Experiments on multiple datasets show that our model produces better results and is substantially more efficient compared to recent generative models.

Author Information

Taesung Park (UC Berkeley)
Jun-Yan Zhu (Adobe, CMU)
Oliver Wang (Adobe Research)
Jingwan Lu (Adobe Research)

Jingwan joined Adobe Research in August 2014. Her current research interests include deep-learning based image editing and generation, sketch-based search, creative applications for AR and VR, data-driven visual content creation, computational photography and other vision and machine learning topics. Jingwan received her Ph.D. in computer science from Princeton University. Her PhD work focused on designing algorithms and interfaces for data-driven painting applications. During her PhD, she was awarded Google Research Fellowship from 2012 to 2014 and Siebel Scholarship from 2013 to 2014.

Eli Shechtman (Adobe Research, US)
Alexei Efros (UC Berkeley)
Richard Zhang (Adobe)

Richard Zhang is a Research Scientist at Adobe Research, with interests in computer vision, deep learning, machine learning, and graphics. He obtained his PhD in EECS, advised by Professor Alexei A. Efros, at UC Berkeley in 2018. He graduated summa cum laude with BS and MEng degrees from Cornell University in ECE. He is a recipient of the 2017 Adobe Research Fellowship. More information can be found on his webpage: http://richzhang.github.io/.

More from the Same Authors