Timezone: »
Mainstream captioning models often follow a sequential structure to generate cap- tions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance. In this paper, we present an alternative paradigm for image captioning, which factorizes the captioning procedure into two stages: (1) extracting an explicit semantic representation from the given image; and (2) constructing the caption based on a recursive compositional procedure in a bottom-up manner. Compared to conventional ones, our paradigm better preserves the semantic content through an explicit factorization of semantics and syntax. By using the compositional generation procedure, caption construction follows a recursive structure, which naturally fits the properties of human language. Moreover, the proposed compositional procedure requires less data to train, generalizes better, and yields more diverse captions.
Author Information
Bo Dai (The Chinese University of Hong Kong)
Sanja Fidler (University of Toronto)
Dahua Lin (The Chinese University of Hong Kong)
More from the Same Authors
-
2022 Poster: Audio-Driven Co-Speech Gesture Video Generation »
Xian Liu · Qianyi Wu · Hang Zhou · Yuanqi Du · Wayne Wu · Dahua Lin · Ziwei Liu -
2022 : MOPA: a Minimalist Off-Policy Approach to Safe-RL »
Hao Sun · Ziping Xu · Zhenghao Peng · Meng Fang · Bo Dai · Bolei Zhou -
2023 Poster: Learning Modulated Transformation in GANs »
Ceyuan Yang · Qihang Zhang · Yinghao Xu · Jiapeng Zhu · Yujun Shen · Bo Dai -
2023 Poster: Revisiting the Evaluation of Image Synthesis with GANs »
mengping yang · Ceyuan Yang · Yichi Zhang · Qingyan Bai · Yujun Shen · Bo Dai -
2023 Poster: RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars »
Dongwei Pan · Long Zhuo · Jingtan Piao · Huiwen Luo · Wei Cheng · Yuxin WANG · Siming Fan · Shengqi Liu · Lei Yang · Bo Dai · Ziwei Liu · Chen Change Loy · Chen Qian · Wayne Wu · Dahua Lin · Kwan-Yee Lin -
2022 : Factor Investing with a Deep Multi-Factor Model »
Zikai Wei · Bo Dai · Dahua Lin -
2022 Spotlight: Lightning Talks 4B-4 »
Ziyue Jiang · Zeeshan Khan · Yuxiang Yang · Chenze Shao · Yichong Leng · Zehao Yu · Wenguan Wang · Xian Liu · Zehua Chen · Yang Feng · Qianyi Wu · James Liang · C.V. Jawahar · Junjie Yang · Zhe Su · Songyou Peng · Yufei Xu · Junliang Guo · Michael Niemeyer · Hang Zhou · Zhou Zhao · Makarand Tapaswi · Dongfang Liu · Qian Yang · Torsten Sattler · Yuanqi Du · Haohe Liu · Jing Zhang · Andreas Geiger · Yi Ren · Long Lan · Jiawei Chen · Wayne Wu · Dahua Lin · Dacheng Tao · Xu Tan · Jinglin Liu · Ziwei Liu · 振辉 叶 · Danilo Mandic · Lei He · Xiangyang Li · Tao Qin · sheng zhao · Tie-Yan Liu -
2022 Spotlight: Audio-Driven Co-Speech Gesture Video Generation »
Xian Liu · Qianyi Wu · Hang Zhou · Yuanqi Du · Wayne Wu · Dahua Lin · Ziwei Liu -
2022 Poster: Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant »
Ying Jin · Jiaqi Wang · Dahua Lin -
2022 Poster: Improving GANs with A Dynamic Discriminator »
Ceyuan Yang · Yujun Shen · Yinghao Xu · Deli Zhao · Bo Dai · Bolei Zhou -
2021 Poster: Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis »
Tianchang Shen · Jun Gao · Kangxue Yin · Ming-Yu Liu · Sanja Fidler -
2021 Poster: Scalable Neural Data Server: A Data Recommender for Transfer Learning »
Tianshi Cao · Sasha (Alexandre) Doubov · David Acuna · Sanja Fidler -
2021 Poster: DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer »
Wenzheng Chen · Joey Litalien · Jun Gao · Zian Wang · Clement Fuji Tsang · Sameh Khamis · Or Litany · Sanja Fidler -
2021 Poster: A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis »
Xingang Pan · Xudong XU · Chen Change Loy · Christian Theobalt · Bo Dai -
2021 Poster: EditGAN: High-Precision Semantic Image Editing »
Huan Ling · Karsten Kreis · Daiqing Li · Seung Wook Kim · Antonio Torralba · Sanja Fidler -
2021 Poster: ATISS: Autoregressive Transformers for Indoor Scene Synthesis »
Despoina Paschalidou · Amlan Kar · Maria Shugrina · Karsten Kreis · Andreas Geiger · Sanja Fidler -
2021 Poster: Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence »
Tianshi Cao · Alex Bie · Arash Vahdat · Sanja Fidler · Karsten Kreis -
2021 Poster: Generative Occupancy Fields for 3D Surface-Aware Image Synthesis »
Xudong XU · Xingang Pan · Dahua Lin · Bo Dai -
2021 Poster: Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion »
Tong Wu · Liang Pan · Junzhe Zhang · Tai WANG · Ziwei Liu · Dahua Lin -
2021 Poster: Few-Shot Object Detection via Association and DIscrimination »
Yuhang Cao · Jiaqi Wang · Ying Jin · Tong Wu · Kai Chen · Ziwei Liu · Dahua Lin -
2021 Poster: Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data »
Liming Jiang · Bo Dai · Wayne Wu · Chen Change Loy -
2021 Poster: Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation »
David Acuna · Jonah Philion · Sanja Fidler -
2020 : Sanja Fidler »
Sanja Fidler -
2020 Poster: Variational Amodal Object Completion »
Huan Ling · David Acuna · Karsten Kreis · Seung Wook Kim · Sanja Fidler -
2020 Poster: Learning Deformable Tetrahedral Meshes for 3D Reconstruction »
Jun Gao · Wenzheng Chen · Tommy Xiang · Alec Jacobson · Morgan McGuire · Sanja Fidler -
2019 : Poster and Coffee Break 2 »
Karol Hausman · Kefan Dong · Ken Goldberg · Lihong Li · Lin Yang · Lingxiao Wang · Lior Shani · Liwei Wang · Loren Amdahl-Culleton · Lucas Cassano · Marc Dymetman · Marc Bellemare · Marcin Tomczak · Margarita Castro · Marius Kloft · Marius-Constantin Dinu · Markus Holzleitner · Martha White · Mengdi Wang · Michael Jordan · Mihailo Jovanovic · Ming Yu · Minshuo Chen · Moonkyung Ryu · Muhammad Zaheer · Naman Agarwal · Nan Jiang · Niao He · Nikolaus Yasui · Nikos Karampatziakis · Nino Vieillard · Ofir Nachum · Olivier Pietquin · Ozan Sener · Pan Xu · Parameswaran Kamalaruban · Paul Mineiro · Paul Rolland · Philip Amortila · Pierre-Luc Bacon · Prakash Panangaden · Qi Cai · Qiang Liu · Quanquan Gu · Raihan Seraj · Richard Sutton · Rick Valenzano · Robert Dadashi · Rodrigo Toro Icarte · Roshan Shariff · Roy Fox · Ruosong Wang · Saeed Ghadimi · Samuel Sokota · Sean Sinclair · Sepp Hochreiter · Sergey Levine · Sergio Valcarcel Macua · Sham Kakade · Shangtong Zhang · Sheila McIlraith · Shie Mannor · Shimon Whiteson · Shuai Li · Shuang Qiu · Wai Lok Li · Siddhartha Banerjee · Sitao Luan · Tamer Basar · Thinh Doan · Tianhe Yu · Tianyi Liu · Tom Zahavy · Toryn Klassen · Tuo Zhao · Vicenç Gómez · Vincent Liu · Volkan Cevher · Wesley Suttle · Xiao-Wen Chang · Xiaohan Wei · Xiaotong Liu · Xingguo Li · Xinyi Chen · Xingyou Song · Yao Liu · YiDing Jiang · Yihao Feng · Yilun Du · Yinlam Chow · Yinyu Ye · Yishay Mansour · · Yonathan Efroni · Yongxin Chen · Yuanhao Wang · Bo Dai · Chen-Yu Wei · Harsh Shrivastava · Hongyang Zhang · Qinqing Zheng · SIDDHARTHA SATPATHI · Xueqing Liu · Andreu Vall -
2019 : Carl Doersch, Raquel Urtasun, Sanja Fidler moderated by Natalia Neverova »
Raquel Urtasun · Sanja Fidler · Natalia Neverova · Ilija Radosavovic · Carl Doersch -
2019 : Sanja Fidler - TBA »
Sanja Fidler -
2019 : Poster and Coffee Break 1 »
Aaron Sidford · Aditya Mahajan · Alejandro Ribeiro · Alex Lewandowski · Ali H Sayed · Ambuj Tewari · Angelika Steger · Anima Anandkumar · Asier Mujika · Hilbert J Kappen · Bolei Zhou · Byron Boots · Chelsea Finn · Chen-Yu Wei · Chi Jin · Ching-An Cheng · Christina Yu · Clement Gehring · Craig Boutilier · Dahua Lin · Daniel McNamee · Daniel Russo · David Brandfonbrener · Denny Zhou · Devesh Jha · Diego Romeres · Doina Precup · Dominik Thalmeier · Eduard Gorbunov · Elad Hazan · Elena Smirnova · Elvis Dohmatob · Emma Brunskill · Enrique Munoz de Cote · Ethan Waldie · Florian Meier · Florian Schaefer · Ge Liu · Gergely Neu · Haim Kaplan · Hao Sun · Hengshuai Yao · Jalaj Bhandari · James A Preiss · Jayakumar Subramanian · Jiajin Li · Jieping Ye · Jimmy Smith · Joan Bas Serrano · Joan Bruna · John Langford · Jonathan Lee · Jose A. Arjona-Medina · Kaiqing Zhang · Karan Singh · Yuping Luo · Zafarali Ahmed · Zaiwei Chen · Zhaoran Wang · Zhizhong Li · Zhuoran Yang · Ziping Xu · Ziyang Tang · Yi Mao · David Brandfonbrener · Shirli Di-Castro · Riashat Islam · Zuyue Fu · Abhishek Naik · Saurabh Kumar · Benjamin Petit · Angeliki Kamoutsi · Simone Totaro · Arvind Raghunathan · Rui Wu · Donghwan Lee · Dongsheng Ding · Alec Koppel · Hao Sun · Christian Tjandraatmadja · Mahdi Karami · Jincheng Mei · Chenjun Xiao · Junfeng Wen · Zichen Zhang · Ross Goroshin · Mohammad Pezeshki · Jiaqi Zhai · Philip Amortila · Shuo Huang · Mariya Vasileva · El houcine Bergou · Adel Ahmadyan · Haoran Sun · Sheng Zhang · Lukas Gruber · Yuanhao Wang · Tetiana Parshakova -
2019 : Panel »
Sanja Fidler · Josh Tenenbaum · Tatiana López-Guevara · Danilo Jimenez Rezende · Niloy Mitra -
2019 : Sanja Fidler »
Sanja Fidler -
2019 Poster: Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer »
Wenzheng Chen · Huan Ling · Jun Gao · Edward Smith · Jaakko Lehtinen · Alec Jacobson · Sanja Fidler -
2019 Poster: Policy Continuation with Hindsight Inverse Dynamics »
Hao Sun · Zhizhong Li · Xiaotong Liu · Bolei Zhou · Dahua Lin -
2019 Spotlight: Policy Continuation with Hindsight Inverse Dynamics »
Hao Sun · Zhizhong Li · Xiaotong Liu · Bolei Zhou · Dahua Lin -
2019 Demonstration: Toronto Annotation Suite »
Amlan Kar · Sanja Fidler · Jun Gao · Seung Wook Kim · Huan Ling -
2018 Poster: Trajectory Convolution for Action Recognition »
Yue Zhao · Yuanjun Xiong · Dahua Lin -
2017 : Panel Discussion »
Felix Hill · Olivier Pietquin · Jack Gallant · Raymond Mooney · Sanja Fidler · Chen Yu · Devi Parikh -
2017 : Connecting high-level semantics with low-level vision »
Sanja Fidler -
2017 Poster: Contrastive Learning for Image Captioning »
Bo Dai · Dahua Lin -
2017 Poster: Teaching Machines to Describe Images with Natural Language Feedback »
Huan Ling · Sanja Fidler -
2016 Poster: Proximal Deep Structured Models »
Shenlong Wang · Sanja Fidler · Raquel Urtasun -
2015 Poster: Skip-Thought Vectors »
Jamie Kiros · Yukun Zhu · Russ Salakhutdinov · Richard Zemel · Raquel Urtasun · Antonio Torralba · Sanja Fidler -
2015 Poster: 3D Object Proposals for Accurate Object Class Detection »
Xiaozhi Chen · Kaustav Kundu · Yukun Zhu · Andrew G Berneshawi · Huimin Ma · Sanja Fidler · Raquel Urtasun -
2013 Poster: Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation »
Dahua Lin -
2012 Poster: Coupling Nonparametric Mixtures via Latent Dirichlet Processes »
Dahua Lin · John Fisher III -
2012 Poster: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2012 Spotlight: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2010 Oral: Construction of Dependent Dirichlet Processes based on Poisson Processes »
Dahua Lin · Eric Grimson · John Fisher III -
2010 Poster: Construction of Dependent Dirichlet Processes based on Poisson Processes »
Dahua Lin · Eric Grimson · John Fisher III -
2009 Poster: Evaluating multi-class learning strategies in a generative hierarchical framework for object detection »
Sanja Fidler · Marko Boben · Ales Leonardis