Expo Talk Panel
Room 215 - 216

"Resonator: Music Space" is a project that connects people through explorations of music. Built by a video game development team at Google, the project lets non-experts see inside the "black box" of an AI model using a 3D video game engine. First, a scaffolded and gamified conversational experience elicits preferences about music, playable with either two human players or one human player and a large language model player. The human participant can then refine these preferences, or jump immediately to the automated creation of a music playlist. This playlist then becomes the starting point of an exploration in digital space. "Music Space" shows a controllable 3D projection of songs provided by YouTube, analyzed by Google's MuLan music understanding joint embedding model. 128-dimensional embeddings are reduced using UMAP or by user-specified term queries to the joint embedding model, rendering songs as "stars" in an explorable galaxy. By translating high dimensional embeddings into 3D space, and making that space explorable using video game controls, we capture the beauty and mystery of AI in an immersive experience, while conveying real analysis that provides transparency into the model representation. Together, the experiences provide a new AI-powered way of exploring and discovering music, while connecting with others on the same journey.

Chat is not available.