( events)   Timezone: »  
Mon Dec 13 08:00 AM -- 12:00 AM (PST)
CtrlGen: Controllable Generative Modeling in Language and Vision
Steven Y. Feng · Dor Arad Hudson · Tatsunori Hashimoto · DONGYEOP Kang · Varun Prashant Gangal · Anusha Balakrishnan · Joel Tetreault

Workshop Home Page

Over the past few years, there has been an increased interest in the areas of language and image generation within the community. As generated texts by models like GPT-3 start to sound more fluid and natural, and generated images and videos by GAN models appear more realistic, researchers began focusing on qualitative properties of the generated content such as the ability to control its style and structure, or incorporate information from external sources into the output. Such aims are extremely important to make language and image generation useful for human-machine interaction and other real-world applications including machine co-creativity, entertainment, reducing biases or toxicity, and improving conversational agents and personal assistants.

Achieving these ambitious but important goals introduces challenges not only from NLP and Vision perspectives, but also ones that pertain to Machine Learning as a whole, which has witnessed a growing body of research in relevant domains such as interpretability, disentanglement, robustness, and representation learning. We believe that progress towards the realization of human-like language and image generation may benefit greatly from insights and progress in these and other ML areas.

In this workshop, we propose to bring together researchers from the NLP, Vision, and ML communities to discuss the current challenges and explore potential directions for controllable generation and improve its quality, correctness, and diversity. As excitement about language and image generation has significantly increased recently thanks to the advent and improvement of language models, Transformers, and GANs, we feel this is the opportune time to hold a new workshop about this subject. We hope CtrlGen will foster discussion and interaction across communities, and sprout fruitful cross-domain relations that open the door for enhanced controllability in language and image generation.

Opening Remarks (Short Intro)
Invited Talk #1 - Jason Weston (Invited Talk)
Invited Talk #1 Q&A (Short Q&A)
Invited Talk #2 - He He (Invited Talk)
Invited Talk #2 Q&A (Short Q&A)
Invited Talk #3 - Irina Higgins (Invited Talk)
Invited Talk #3 Q&A (Short Q&A)
Invited Talk #4 - Yejin Choi (Invited Talk)
Invited Talk #4 Q&A (Short Q&A)
Virtual Coffee/Networking Break (Break)
Discussion Panel and QA Session (Discussion Panel)
Virtual Poster Session #1 (Poster Session)
Lunch Break (Break)
Demonstrations or Interactive Activity
Invited Talk #5 - Alex Tamkin (Invited Talk)
Invited Talk #5 Q&A (Short Q&A)
Invited Talk #6 - Or Patashnik (Invited Talk)
Invited Talk #6 Q&A (Short Q&A)
Virtual Coffee/Networking Break (Break)
Virtual Poster Session #2 (Poster Session)
Demonstrations or Interactive Activity
Invited Talk #7: Controllable Text Generation with Multiple Constraints (Yulia Tsvetkov) (Invited Talk)
Invited Talk #7 Q&A (Short Q&A)
Best Paper Awards and Closing Remarks (Closing Remarks)
GatherTown Open for Continued Socializing (Networking and Socializing)
MIDI-DDSP: Hierarchical Modeling of Music for Detailed Control (Poster)
C^3: Contrastive Learning for Cross-domain Correspondence in Few-shot Image Generation (Poster)
Robust Text Generation using Sequence-to-Sequence Pre-Training (Poster)
Fair Data Generation using Language Models with Hard Constraints (Poster)
Controllable Paraphrase Generation with Multiple Types of Constraints (Poster)
Self-supervised Enhancement of Latent Discovery in GANs (Poster)
Diamond in the rough: Improving image realism by traversing the GAN latent space (Poster)
Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs (Poster)
Controlled Cue Generation for Play Scripts (Poster)
Hamiltonian prior to Disentangle Content and Motion in Image Sequences (Poster)
Continuous Emotion Transfer Using Kernels (Poster)
Learning Representations for Zero-Shot Image Generation without Text (Poster)
Learning to Compose Visual Relations (Poster)
Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning (Poster)
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges (Poster)
Controlling Conditional Language Models with Distributional Policy Gradients (Poster)
Sound-Guided Semantic Image Manipulation (Poster)
Towards Unsupervised Content Disentanglement in Sentence Representations via Syntactic Roles (Poster)
PPL-MCTS: Constrained Textual Generation Through Discriminator-Guided Decoding (Poster)
XCI-Sketch: Extraction of Color Information from Images for Generation of Colored Outlines and Sketches (Poster)