Timezone: »
Animal pose estimation and tracking (APT) is a fundamental task for detecting and tracking animal keypoints from a sequence of video frames. Previous animal-related datasets focus either on animal tracking or single-frame animal pose estimation, and never on both aspects. The lack of APT datasets hinders the development and evaluation of video-based animal pose estimation and tracking methods, limiting the applications in real world, e.g., understanding animal behavior in wildlife conservation. To fill this gap, we make the first step and propose APT-36K, i.e., the first large-scale benchmark for animal pose estimation and tracking. Specifically, APT-36K consists of 2,400 video clips collected and filtered from 30 animal species with 15 frames for each video, resulting in 36,000 frames in total. After manual annotation and careful double-check, high-quality keypoint and tracking annotations are provided for all the animal instances. Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking. Based on the experimental results, we gain some empirical insights and show that APT-36K provides a useful animal pose estimation and tracking benchmark, offering new challenges and opportunities for future research. The code and dataset will be made publicly available at https://github.com/pandorgan/APT-36K.
Author Information
Yuxiang Yang (Hangzhou Dianzi University)
Junjie Yang (Hangzhou Dianzi University)
Yufei Xu (The University of Sydney, University of Sydney)
Jing Zhang (The University of Sydney)
Long Lan (National University of Defense Technology, Tsinghua University)
Dacheng Tao (University of Technology, Sydney)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking »
Dates n/a. Room
More from the Same Authors
-
2021 Spotlight: TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation »
Haoang Chi · Feng Liu · Wenjing Yang · Long Lan · Tongliang Liu · Bo Han · William Cheung · James Kwok -
2021 : AP-10K: A Benchmark for Animal Pose Estimation in the Wild »
Hang Yu · Yufei Xu · Jing Zhang · Wei Zhao · Ziyu Guan · Dacheng Tao -
2022 Poster: Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach »
Peng Mi · Li Shen · Tianhe Ren · Yiyi Zhou · Xiaoshuai Sun · Rongrong Ji · Dacheng Tao -
2022 Poster: Exploring Figure-Ground Assignment Mechanism in Perceptual Organization »
Wei Zhai · Yang Cao · Jing Zhang · Zheng-Jun Zha -
2022 Poster: ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation »
Yufei Xu · Jing Zhang · Qiming ZHANG · Dacheng Tao -
2023 Poster: Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initialization »
Yan Sun · Li Shen · Dacheng Tao -
2023 Poster: SODA: Robust Training of Test-Time Data Adaptors »
Zige Wang · Yonggang Zhang · Zhen Fang · Long Lan · Wenjing Yang · Bo Han -
2023 Poster: All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation »
Liyao Tang · Zhe Chen · Shanshan Zhao · Chaoyue Wang · Dacheng Tao -
2023 Poster: Cocktail: Mixing Multi-Modality Control for Text-Conditional Image Generation »
Minghui Hu · Jianbin Zheng · Daqing Liu · Chuanxia Zheng · Chaoyue Wang · Dacheng Tao · Tat-Jen Cham -
2023 Poster: Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm »
Miaoxi Zhu · Li Shen · Bo Du · Dacheng Tao -
2023 Poster: ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding »
Lunhao Duan · Shanshan Zhao · Nan Xue · Mingming Gong · Gui-Song Xia · Dacheng Tao -
2023 Poster: Domain Re-Modulation for Few-Shot Generative Domain Adaptation »
Yi Wu · Ziqiang Li · Chaoyue Wang · Heliang Zheng · Shanshan Zhao · Bin Li · Dacheng Tao -
2023 Poster: Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning »
Guozheng Ma · Linrui Zhang · Haoyu Wang · Lu Li · Zilin Wang · Zhen Wang · Li Shen · Xueqian Wang · Dacheng Tao -
2023 Poster: Towards Arbitrarily Expressive GNNs in $O(n^2)$ Space by Rethinking Folklore Weisfeiler-Lehman »
Jiarui Feng · Lecheng Kong · Hao Liu · Dacheng Tao · Fuhai Li · Muhan Zhang · Yixin Chen -
2023 Poster: VanillaNet: the Power of Minimalism in Deep Learning »
Hanting Chen · Yunhe Wang · Jianyuan Guo · Dacheng Tao -
2023 Poster: MAG-GNN: Reinforcement Learning Boosted Graph Neural Network »
Lecheng Kong · Jiarui Feng · Hao Liu · Dacheng Tao · Yixin Chen · Muhan Zhang -
2023 Poster: SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model »
Di Wang · Jing Zhang · Bo Du · Minqiang Xu · Lin Liu · Dacheng Tao · Liangpei Zhang -
2022 Spotlight: Lightning Talks 5B-3 »
Yanze Wu · Jie Xiao · Nianzu Yang · Jieyi Bi · Jian Yao · Yiting Chen · Qizhou Wang · Yangru Huang · Yongqiang Chen · Peixi Peng · Yuxin Hong · Xintao Wang · Feng Liu · Yining Ma · Qibing Ren · Xueyang Fu · Yonggang Zhang · Kaipeng Zeng · Jiahai Wang · GEN LI · Yonggang Zhang · Qitian Wu · Yifan Zhao · Chiyu Wang · Junchi Yan · Feng Wu · Yatao Bian · Xiaosong Jia · Ying Shan · Zhiguang Cao · Zheng-Jun Zha · Guangyao Chen · Tianjun Xiao · Han Yang · Jing Zhang · Jinbiao Chen · MA Kaili · Yonghong Tian · Junchi Yan · Chen Gong · Tong He · Binghui Xie · Yuan Sun · Francesco Locatello · Tongliang Liu · Yeow Meng Chee · David P Wipf · Tongliang Liu · Bo Han · Bo Han · Yanwei Fu · James Cheng · Zheng Zhang -
2022 Spotlight: Escaping from the Barren Plateau via Gaussian Initializations in Deep Variational Quantum Circuits »
Kaining Zhang · Liu Liu · Min-Hsiu Hsieh · Dacheng Tao -
2022 Spotlight: Watermarking for Out-of-distribution Detection »
Qizhou Wang · Feng Liu · Yonggang Zhang · Jing Zhang · Chen Gong · Tongliang Liu · Bo Han -
2022 Spotlight: Lightning Talks 4B-4 »
Ziyue Jiang · Zeeshan Khan · Yuxiang Yang · Chenze Shao · Yichong Leng · Zehao Yu · Wenguan Wang · Xian Liu · Zehua Chen · Yang Feng · Qianyi Wu · James Liang · C.V. Jawahar · Junjie Yang · Zhe Su · Songyou Peng · Yufei Xu · Junliang Guo · Michael Niemeyer · Hang Zhou · Zhou Zhao · Makarand Tapaswi · Dongfang Liu · Qian Yang · Torsten Sattler · Yuanqi Du · Haohe Liu · Jing Zhang · Andreas Geiger · Yi Ren · Long Lan · Jiawei Chen · Wayne Wu · Dahua Lin · Dacheng Tao · Xu Tan · Jinglin Liu · Ziwei Liu · 振辉 叶 · Danilo Mandic · Lei He · Xiangyang Li · Tao Qin · sheng zhao · Tie-Yan Liu -
2022 Spotlight: Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach »
Kaiwen Yang · Yanchao Sun · Jiahao Su · Fengxiang He · Xinmei Tian · Furong Huang · Tianyi Zhou · Dacheng Tao -
2022 Poster: Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network? »
Yibo Yang · Shixiang Chen · Xiangtai Li · Liang Xie · Zhouchen Lin · Dacheng Tao -
2022 Poster: CGLB: Benchmark Tasks for Continual Graph Learning »
Xikun Zhang · Dongjin Song · Dacheng Tao -
2022 Poster: Watermarking for Out-of-distribution Detection »
Qizhou Wang · Feng Liu · Yonggang Zhang · Jing Zhang · Chen Gong · Tongliang Liu · Bo Han -
2022 Poster: Escaping from the Barren Plateau via Gaussian Initializations in Deep Variational Quantum Circuits »
Kaining Zhang · Liu Liu · Min-Hsiu Hsieh · Dacheng Tao -
2022 Poster: Benefits of Permutation-Equivariance in Auction Mechanisms »
Tian Qin · Fengxiang He · Dingfeng Shi · Wenbing Huang · Dacheng Tao -
2022 Poster: Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach »
Kaiwen Yang · Yanchao Sun · Jiahao Su · Fengxiang He · Xinmei Tian · Furong Huang · Tianyi Zhou · Dacheng Tao -
2021 Poster: Class-Disentanglement and Applications in Adversarial Detection and Defense »
Kaiwen Yang · Tianyi Zhou · Yonggang Zhang · Xinmei Tian · Dacheng Tao -
2021 Poster: Gauge Equivariant Transformer »
Lingshen He · Yiming Dong · Yisen Wang · Dacheng Tao · Zhouchen Lin -
2021 Poster: ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias »
Yufei Xu · Qiming ZHANG · Jing Zhang · Dacheng Tao -
2021 Poster: TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation »
Haoang Chi · Feng Liu · Wenjing Yang · Long Lan · Tongliang Liu · Bo Han · William Cheung · James Kwok -
2018 Poster: Learning Versatile Filters for Efficient Convolutional Neural Networks »
Yunhe Wang · Chang Xu · Chunjing XU · Chao Xu · Dacheng Tao