Cadence: Music-Responsive Stage Lighting in Concert Video Generation

Stage lighting brings music to life, turning sound into an immersive visual experience. Yet designing responsive lighting cues demands hours of tedious manual work from lighting designers. Existing audio-to-video models capture human movement or overall visual mood, but overlook the art of dynamic lighting. To bridge this gap, we first introduce LuSyClip, a concert lighting video benchmark for training and evaluating music-responsive stage lighting generation. We then propose Cadence, a framework that generates concert video whose stage lighting follows the music. Cadence builds on a state-of-the-art video backbone and adds three trainable components: (i) a music-relative aligner that contrasts the response to the music with the response to silence, deciding what should change; (ii) a temporal control that decides when the change acts; and (iii) a spatial control that decides where it may act. On 1,032 held-out clips, relative to a fine-tuned baseline, Cadence reduces distributional distance by 28 % and increases lighting activity by 44.2 %, moving generated activity towards the recorded level. Together, these contributions give lighting designers a responsive and controllable tool for stage lighting design.