Rhythm: A Consumer Wearable Ear-Worn Platform for Real-Time Multimodal Biosignal Decoding and Closed-Loop Interaction

Demo Description Foundation models for biosignals need large, multimodal datasets collected under real-world conditions, yet most recording hardware is research-grade and stays in the lab. We introduce Rhythm, an ear-worn device designed to be worn all day while recording research-quality biosignals. It hooks around the ear with sensor contacts at the tragus/temporal region and the mastoid process, and is designed to look and feel good while providing surface area for sensor arrays and actuators. It weighs 25 g, measures [70 × 60 × 7 mm], and runs for 8h on a single charge.

Rhythm is a modular wearable sensor platform. The current prototype contains bone-conduction microphones, and an array of actuators and sensors (e.g., EMG/EEG) that allow the device to sense, decode, and respond to the wearer. We designed our system with privacy in mind: all decoding algorithms run locally, and the biosignals themselves never leave the laptop. On-device compute is planned for the next prototype revision. We believe this combination of an all-day wearable form factor, multimodal sensing, and privacy-preserving capture is what makes informed data collection at scale realistic; our dataset currently stands at ~100 hours with active, ongoing data collection.

Attendee experience (about five minutes per participant): 1. Enrolment. The participant puts on the Rhythm device and enrols in ~30 seconds via an acoustic response signal which is unique to them. 2. Live streaming. EEG/EMG, PPG, audio, and IMU data stream in real time to a large monitor. 3. Closed-loop decoding. The participant speaks a short command to bring up accept/reject options, then confirms with an "mm-hmm" or dismisses with a "tsk" (alternatively, a jaw or tongue movement). And participants receive both a verbal and vibrotactile response. Bone-conduction input and output keeps the audio interaction robust and private in a noisy demo environment. 4. Authentication. A second participant puts on the same device and is rejected.

System Description Proprietary hardware streams over Bluetooth to a laptop. Our pipeline handles decoding and full-fidelity recording to MCAP, an open container format readable by a range of open-source tools and customised live visualization.

Demo Goals We aim to demonstrate the capabilities of wearable biosignal data from hardware designed for all-day wear: real-time decoding from a consumer form factor, multimodal capture suited to foundation-model training, and closed-loop interaction between the wearer and machine. We believe this is well aligned with the embodied, real-world direction this workshop aims to highlight.