NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamics

Physically consistent 3D object dynamics is a key component for controllable simulation and physics-grounded video generation, especially under contact, deformation, and external forcing. Existing trajectory-based methods enable physical control, but they often model isolated physical effects rather than deriving motion from a unified physical structure. This makes it difficult to compose conservative and non-conservative effects in contact-rich 3D dynamics while retaining explicit control. We present NEXUS, a neural energy-field method for contact-rich 3D object dynamics. NEXUS represents each object as a structural graph and constructs dynamic contact graphs for interactions. Inspired by the Hamiltonian Neural Network (HNN), NEXUS formulates dynamics through scalar energy and dissipation terms rather than direct state or acceleration regression. Conservative effects are composed as additive energy terms over the scene. To handle non-conservative systems beyond standard HNNs, NEXUS learns a dissipation function for impact-induced energy loss. Forces are derived by differentiating the energy-dissipation functions and rolled out with a numerical integrator. NEXUS provides a stable and controllable physics reasoning module for contact-rich 3D object dynamics, improving long-horizon accuracy over representative learned and physics-structured dynamics baselines across controlled rollouts with varying mechanical properties and physical-effect compositions. Our contributions are: (i) an HNN-inspired energy-dissipation dynamics formulation that unifies conservative and non-conservative scene effects; (ii) a graph-based 3D representation for contact-rich object dynamics, with object--object contact evaluated in mixed-contact stress tests; and (iii) a trajectory-guided video generation study showing that physically consistent motion improves downstream physical plausibility while maintaining competitive visual quality.