Depression and anxiety characterization and detection with multimodal deep learning

Abstract

Depression and anxiety are among the most prevalent mental disorders, necessitating accurate characterization for effective diagnosis and treatment. Multimodal deep learning has emerged as an effective approach to enhance diagnostic precision by integrating diverse data sources, including electronic health records, physiological signals and neuroimaging. This Review provides an overview of the recent advancements in multimodal deep learning for depression and anxiety estimation. Key neural network architectures—such as convolutional neural networks for image analysis, recurrent and transformer models for sequential and textual data, and graph neural networks for capturing complex neuroimaging connectivity patterns—are examined. Challenges in data fusion, feature extraction and model interpretability are discussed, alongside strategies to improve generalizability through transfer learning. Future challenges and opportunities are discussed: large-scale datasets, standardized evaluation protocols and interdisciplinary collaboration to bridge the gap between multimodal deep learning and clinical relevance. By summarizing current practices and identifying critical challenges, this Review highlights the transformative potential of multimodal deep learning in advancing the characterization and detection of depression and anxiety. © Springer Nature America, Inc. 2026.