2026-10-12T04:44:47Z https://www.scidb.cn/oai

10.57760/sciencedb.27127 2025-09-04T12:35:07Z

2025-09-04 Dual-Teacher Multimodal Fusion for Depression Detection doi:10.57760/sciencedb.27127 en Depression affects over 264 million people worldwide and is a leading cause of disability. Existing machine learning approaches for analyzing behavioral data—including electronic health records, wearable devices, and social media—remain constrained by unimodal paradigms. Language models often ignore paralinguistic biomarkers such as speech prosody, while audio-based methods overlook semantic context, leaving critical cross-modal interactions unexploited. Here, we introduce a Teacher-Student Architecture-based Multimodal Fusion Network (TSA-MFN) that fundamentally advances depression classification. Dual specialized teacher models extract high-confidence text semantics and audio biomarkers, which are integrated by a student network through multi-head attention mechanisms. Fusion weights are dynamically computed via learnable similarity matrices, and a hybrid loss function balances knowledge transfer with classification optimization, enabling phased teacher-student synergy during training. TSA-MFN achieves a 99.1% F1-score on the DAIC-WOZ dataset, surpassing unimodal baselines by 18.7% and conventional fusion models by 12.4%, with ablation studies confirming the contribution of each component. By combining dual-teacher guidance with adaptive similarity-based fusion, this framework establishes a new paradigm for interpretable and balanced multimodal learning in mental health analytics. Multimodal fusion; LLM; Teacher-student architecture; Depression detection; Machine learning based health-care Lin Gan PUBLIC https://api.github.com/licenses/cc0-1.0 dataset Science Data Bank