NeurIPS 2022

Timezone: »

Competition

MyoChallenge: Learning contact-rich manipulation using a musculoskeletal hand

Vittorio Caggiano · · Guillaume Durandau · Seungmoon Song · Yuval Tassa · Massimo Sartori · Vikash Kumar

Tue Dec 06 01:00 PM -- 04:00 PM (PST) @ Virtual

Event URL: https://sites.google.com/view/myochallenge »

Manual dexterity has been considered one of the critical components for human evolution. The ability to perform movements as simple as holding and rotating an object in the hand without dropping it needs the coordination of more than 35 muscles which, act synergistically or antagonistically on multiple joints. They control the flexion and extension of the joints connecting the bones which in turn allow the manipulation to happen. This complexity in control is markedly different than typical pre-specified movements or torque based controls used in robotics. In this competition - MyoChallenge, participant will develop controllers for a realistic hand to solve a series of dexterous manipulation tasks. Participant will be provided with a physiologically accurate and efficient neuromusculoskeletal human hand model developed in the (free) MuJoCo physics simulator. In addition the provided model has also contact rich capabilities. Participant will be interfacing with a standardized training environment to help build the controllers. The final score will then be based on a environment with unknown parameters.This challenge builds on 3 previous NeurIPS challenge on controlling legs mus- culoskeletal model for locomotion, which attracted about 1300 participants and generated 8000 submissions, which produced 9 academic publications. This chal- lenge will leverage the experience and knowledge from the previous challenges and will further establish neuromusculoskeletal modelling as a benchmarks for the neuromuscular control and machine learning community.In addition of providing challenges for the biomechanics and machine learning community, this challenge will provide new opportunities to explore solutions that will inspire the robotic, medical and rehabilitation fields on one of the most complex dexterous skills humans are able to perform.

Tue 1:00 p.m. - 1:05 p.m.	Intro to the workshop ( Intro )	Guillaume Durandau 🔗
Tue 1:05 p.m. - 1:10 p.m.	MyoSuite introduction ( Presentation )	🔗
Tue 1:10 p.m. - 1:15 p.m.	MyoChallenge results and future ( Presentation )	Vittorio Caggiano 🔗
Tue 1:15 p.m. - 1:35 p.m.	How to Make Biomechanical Constraints Work in Your Favor ( Presentation ) Biomechanical constraints are often perceived as a nuisance: they add an extra layer of complexity to an already complex motor control problem. However, the musculoskeletal and nervous systems have been evolving in conjunction for hundreds of millions of years, and are heavily optimized to work well together as a result. Comprehensive neuromusculoskeletal modeling has the potential to exploit this phenomenon, and drastically simplify and improve motor control learning.	🔗
Tue 1:35 p.m. - 1:55 p.m.	Modeling sensorimotor circuits with task-driven and reinforcement learning ( Presentation ) Biological adaptive motor control relies on the integration of proprioception and hierarchical control. To illustrate our research on those topics, I will firstly, present a task-driven modeling approach to quantitatively test hypotheses about the functional role of proprioceptive neurons in the brain stem and cortex. Secondly, I will discuss DMAP, a biologically-inspired, attention-based policy network architecture that can learn to walk with changing bodies (Chiappa et al., NeurIPS 2022).	🔗
Tue 1:55 p.m. - 2:00 p.m.	Break	🔗
Tue 2:00 p.m. - 2:20 p.m.	Physiological Mapping of the Muscle Electrical Activity into Behaviour: Principles of Neuromechanics, High-Density Electromyographic Systems, and New Decoding Algorithms ( Presentation ) One of the biggest challenges faced by neuroscience is to have robust neuro sensing methods that translate neural activity into stable movement dynamics and with minimally invasive methods. The spinal motor neurons, the muscles, and the tendons constitute a fundamental interface that allows the nervous system to communicate with the environment in a stable and reliable way. Here I will present and discuss recent data and new decoding algorithms that identify the latent embeddings controlling all of the functional degrees of freedom of the human hand through a feedforward neural network that maps the activity of hundreds of electromyographic sensors placed on the extrinsic hand muscles. This will be followed with a view on the neural control of the muscles from the firings of populations of individual human motor neurons during a vast range of hand movements and lower limb motor actions.	🔗
Tue 2:20 p.m. - 2:40 p.m.	A Musculoskeletal Model of the Hand and Wrist Capable of Simulating Functional Tasks ( Presentation ) TBA	🔗
Tue 2:40 p.m. - 2:55 p.m.	Winner Boading Ball ( Presentation ) Team stiff_finger	Nisheet Patel 🔗
Tue 2:55 p.m. - 3:00 p.m.	Break Break	🔗
Tue 3:00 p.m. - 3:15 p.m.	Co-winner Die Rotation ( Presentation ) Team pkumarl (Yiran Geng & Boshi An, Peking University)	🔗
Tue 3:15 p.m. - 3:30 p.m.	Co-winner Die Rotation ( Presentation ) Team IARAI-JKU (Rahul Siripurapu, Institute of Advanced Research in Artificial Intelligence )	Rahul Siripurapu 🔗
Tue 3:30 p.m. - 4:00 p.m.	Panel discussion Panel discussion	🔗
Tue 4:00 p.m. - 4:00 p.m.	Closing remark	Guillaume Durandau 🔗
-	Fifteen-minute Competition Overview Video ( Overview ) SlidesLive Video »	Guillaume Durandau · Yuval Tassa · Vittorio Caggiano · Vikash Kumar · Seungmoon Song · Massimo Sartori · 🔗

Author Information

Vittorio Caggiano (Meta AI)

Guillaume Durandau (University of Twente)

Guillaume Durandau is a postdoctoral researcher at the Department of Biomechanical Engineering, University of Twente, Enschede, the Netherlands.

Seungmoon Song (Stanford University)

Yuval Tassa (Google DeepMind)

Massimo Sartori (University of Twente)

Vikash Kumar (FAIR, Meta-AI)

I am currently a research scientist at Facebook AI Research (FAIR). I have also spent some time at Google-Brain, OpenAI and Berkeley Artificial Intelligence Research (BAIR) Lab. I did my PhD at CSE, University of Washington's Movement Control Lab, under the supervision of Prof. Emanuel Todorov and Prof. Sergey Levine. I am interested in the areas of Robotics, and Embodied Artificial Intelligence. My general interest lies in developing artificial agents that are cheap, portable and exhibit complex behaviors.