Mohammad Soleymanpour, Yeojeong Kim, Kenneth Cho, and Vijendra Raj Apsingekar, "Test-Time Adaptation with Neural Memory for Dysarthric Speech Recognition"
Abstract
Dysarthric speech remains a hard case for ASR because acoustic variability depends on the speaker, the severity and the session, and little labeled data is available for adapting to each speaker. We study test-time adaptation (TTA) of a general-purpose Conformer-CTC model on TORGO, with no training on dysarthric data. We compare a frozen baseline with SUTA, DSUTA and FS-MaC (fast–slow memory as context), a memory-augmented method that adapts a small Titans-style memory inside one encoder layer while every backbone parameter stays frozen. All three methods lower word error rate (WER) for dysarthric and typical speakers on both microphones. FS-MaC gives the lowest WER for every speaker, with relative reductions of 13–14% for dysarthric speakers, more than twice the gain of SUTA (4.5–5.7%) and roughly twice that of DSUTA (5.9–7.4%). The benefit, however, depends on severity and utterance type. From mild to severe speakers, FS-MaC’s WER gain falls from 28.0% to 10.8% and its character error rate (CER) gain falls from 24.0% to 1.9%. SemScore rises by 4.9 points for mild speakers but drops by 2.6 points for severe speakers. On dysarthric multi-word utterances FS-MaC improves WER, CER and SemScore together.