2026-10-12T12:45:31Z https://www.scidb.cn/oai

10.57760/sciencedb.18722 2025-04-14T09:39:50Z

2025-04-14 AUSpeech: An Audio-Ultrasound Synchronized Database of Tongue Movement for Mandarin speech doi:10.57760/sciencedb.18722 en Ultrasound has been used to render animations of articulation during speech production, specifically tongue movement, for visual feedback used in intervention for articulation disorders and speech recognition. Nevertheless, the availability of high-quality audio-ultrasound datasets remains scarce. The present study, therefore, aims to construct a multimodal database designed for Mandarin speech. The dataset integrates synchronized ultrasound images of lingual movement, and the corresponding audio recordings and text annotations elicited from 43 healthy speakers and 11 patients with dysarthria through speech tasks (including monophthong vowels, monosyllables, and sentences), with a total duration of 22.31 hours. During production, a high-resolution (920×700 at 60 fps) ultrasound device and a high-fidelity microphone were used to simultaneously record tongue motion and audio signals, maintaining sync via experimental setup. In addition, a customized helmet structure was employed to stabilize the ultrasound probe, precisely controlling for head movement, minimizing displacement interference, and ensuring spatial stability of the images. The proposed database carries apparent values in automatic speech recognition, silent interface development, and research in speech pathology and linguistics. Articulatory; Ultrasound Tongue Imaging; Multi-modal Mandarin Speech Datasets Yudong Yang Rongfeng Su Shaofeng Zhao Jianguo Wei Manwa Lawrence Ng Nan Yan Lan Wang RESTRICTED dataset http://www.doi.org/10.1038/s41597-025-04917-w Science Data Bank