Jinghao Chen, Michael Chan, Sicheng Jin, Mostafa Shahin and Beena Ahmed, "Error-Aware Regression-Guided Fine-Tuning for Dysarthric Speech Recognition"
Abstract
Dysarthric automatic speech recognition (ASR) remains challenging due to limited training data and substantial variability across speakers and severity levels. Recent studies show that adapting large pretrained ASR models can substantially improve dysarthric speech recognition. However, recognition difficulty remains highly uneven across utterances and speakers, motivating training strategies that emphasise difficult samples. Existing approaches typically characterise difficulty using predefined grouping or weighting schemes that do not track how recognition difficulty changes as the ASR model evolves. In this work, we propose a regression-guided dysarthric ASR framework that dynamically estimates utterance-level character error rate (CER) and uses the predicted difficulty to weight ASR fine-tuning. A closed-loop update mechanism refreshes the regressor using current recognition errors throughout training. Experiments on the validation split show that our method reduces overall CER by 18.79% and word error rate (WER) by 16.17% relative to the standard fine-tuning baseline.