Vansh Nawander, Chaithra Reddy Nerella, and Manish Shrivastava, "ASR for dysarthric speech in the Speech Accessibility Project Challenge"
Abstract
Despite significant progress in automatic speech recognition, systems trained on typical speech perform poorly on dysarthric speech. Real-world assistive use adds a further constraint, requiring systems to run on a CPU and respond with low latency. The Speech Accessibility Project Challenge 2 (SAPC2) addresses this gap through two tracks, one focusing on recognition accuracy and the other evaluating streaming systems on both accuracy and latency. We investigate both tracks using models ranging from 1.06M to 2B parameters. We first evaluate pretrained models, then improve their accuracy by fine-tuning and reward-based training. For the streaming track, we develop compact models by progressively compressing Moonshine Tiny and training small CTC-based architectures. Fine-tuning reduces Cohere Transcribe’s word error rate (WER) from 18.86% to 8.41%, with further training yielding little improvement. Moonshine Tiny achieves 16.68% WER but compressing it to 2.03M parameters increases WER to 88.37%. The 5.87M and 1.06M streaming CTC models achieve WERs of 53.31% and 67.74%, respectively. Although the 5.87M model processes a 100 ms audio chunk in 2.4 ms, its first word takes a median of 698 ms to stabilize. These results show that reducing model size below 10M parameters leads to substantial degradation in accuracy. our models can be found here: https://huggingface.co/collections/vanshnawander/streaming-dysarthric-asr