Timezone: »

 
Poster
Learning Hierarchical Semantic Image Manipulation through Structured Representations
Seunghoon Hong · Xinchen Yan · Thomas Huang · Honglak Lee

Tue Dec 04 02:00 PM -- 04:00 PM (PST) @ Room 210 #62
in Tue Poster Session B »

Understanding, reasoning, and manipulating semantic concepts of images have been a fundamental research problem for decades. Previous work mainly focused on direct manipulation of natural image manifold through color strokes, key-points, textures, and holes-to-fill. In this work, we present a novel hierarchical framework for semantic image manipulation. Key to our hierarchical framework is that we employ structured semantic layout as our intermediate representations for manipulation. Initialized with coarse-level bounding boxes, our layout generator first creates pixel-wise semantic layout capturing the object shape, object-object interactions, and object-scene relations. Then our image generator fills in the pixel-level textures guided by the semantic layout. Such framework allows a user to manipulate images at object-level by adding, removing, and moving one bounding box at a time. Experimental evaluations demonstrate the advantages of the hierarchical manipulation framework over existing image generation and context hole-filing models, both qualitatively and quantitatively. Benefits of the hierarchical framework are further demonstrated in applications such as semantic object manipulation, interactive image editing, and data-driven image manipulation.

Keywords: Computer Vision
[ Paper] 

Author Information

Seunghoon Hong (University of Michigan)
Xinchen Yan (University of Michigan)
Thomas Huang (University of Michigan)
Honglak Lee (Google / U. Michigan)

More from the Same Authors