Lightning Pose: software for 2D and 3D animal pose estimation

The Lightning Pose App is a javascript- and python-based user interface that supports the full life cycle of animal pose estimation, from data annotation to model training to video inference to diagnostic visualization. The app, which can be run locally or in the cloud, allows users with no coding expertise to estimate animal pose using any computer with access to the internet. The app is based on the Lightning Pose software package, which utilizes semi-supervised learning and a novel Bayesian ensembling technique for post-processing, which together provide improved accuracy and uncertainty estimates with fewer labeled frames. The demo will be run on a local laptop as well as a cloud instance to demonstrate both modes of operation. The demo will run with an existing dataset and allow participants to extract and label new frames, retrain models, and visualize outputs. The goal of the demo is to showcase a software tool designed for practitioners that seamlessly integrates a range of functions. This demo is relevant to the workshop themes of large-scale video analysis, as pose estimation is a foundational technique in this field.

The Lightning Pose App is a comprehensive, user-friendly platform that streamlines the entire 2D and 3D animal pose estimation workflow. Built with JavaScript and Python, this interface guides users through every stage of the process: from annotating data and training models to running video inference and visualizing diagnostic results.

Accessibility and Deployment. The app can run both locally and in the cloud, making animal pose estimation accessible to users without coding expertise. All they need is a computer with internet access. This flexibility ensures researchers can work in their preferred environment while maintaining full functionality.

Advanced Technology. At its core, the app leverages the Lightning Pose software package, which combines semi-supervised learning with a novel Bayesian ensembling technique for post-processing. This innovative approach delivers improved accuracy and more reliable uncertainty estimates while requiring fewer manually labeled frames, a significant advantage for researchers working with limited annotated data.

Interactive Demonstration. Our demo will showcase both operational modes by running on a local laptop instance and a cloud instance simultaneously. Using an existing dataset, participants will experience the complete workflow: extracting and labeling new frames, retraining models, and exploring output visualizations. This hands-on experience will highlight how the tool seamlessly integrates multiple functions into a cohesive platform designed for practitioners. This demonstration directly supports the workshop's focus on large-scale video analysis, as pose estimation serves as a foundational technique in this rapidly growing field.