Timezone: »

 
Assisted Learning of New Languages via User Defined and Phoneme Parameterized Pronunciations
Siddha Ganju · Steven Dalton

@
in Black in AI Workshop »

The ability to notice mispronunciations is a key skill for second language learners. Unfortunately, it usually difficult for learners to acquire consistent feedback concerning their current level of speaking and listening skill. This issue is exacerbated by the fact that standard systems usually accept one correct pronunciation, but humans can understand a larger range of pronunciations which include mispronunciations. Current ASR systems are able to recognize speech almost perfectly, but often don't work when the speaker is not native to that language, i.e. is not spot on with their pronunciation, and more importantly, they don't provide feedback to the user on how to move from the mispronunciation to a more understandable pronunciation, when detected. We propose an approach to detect mispronunciations using a Siamese Network that is trained to recognize not a single correct pronunciation but instead a range of pronunciations that are user defined. The user can control the range of tolerance within which a word is understandable and is based on phoneme pronunciations. As the user interacts, they can adjust the range of tolerance based on their current need.

Author Information

Siddha Ganju (Nvidia)

Siddha Ganju, an AI researcher who Forbes featured in their 30 under 30 list, is a Self-Driving Architect at Nvidia. As an AI Advisor to NASA FDL, she helped build an automated meteor detection pipeline for the CAMS project at NASA, which ended up discovering a comet. Previously at Deep Vision, she developed deep learning models for resource constraint edge devices. Her work ranges from Visual Question Answering to Generative Adversarial Networks to gathering insights from CERN's petabyte-scale data and has been published at top-tier conferences including CVPR and NeurIPS. She has served as a featured jury member in several international tech competitions including CES. As an advocate for diversity and inclusion in technology, she speaks at schools and colleges to motivate and grow a new generation of technologies from all backgrounds. She is also the author of O'Reilly's Practical Deep Learning for Cloud, Mobile and Edge.

Steven Dalton (Nvidia)

More from the Same Authors