A long-standing problem in the design of machine-learning tools for particle physics applications has been how to incorporate prior knowledge of physical symmetries. In this note we propose contrastive self-supervision as a solution to this problem, with jet physics as an example. Using a permutation-invariant transformer network, we learn a representation which outperforms hand-crafted competitors on a linear classification benchmark.