I'll describe a series of studies that use graph networks to reason about and interact with complex physical systems. These models can be used to predict the motion of bodies in particle systems, infer hidden physical properties, control simulated robotic systems, build physical structures, and interpret the symbolic form of the underlying laws that govern physical systems. More generally, this work underlines graph neural networks' role as a first-class member of the deep learning toolkit.