High-Density Electroencephalography and Speech Signal based Deep Framework for Clinical Depression Diagnosis

Abstract

Depression is a mental disorder characterized by persistent depressed mood or loss of interest in performing activities, causing significant impairment in daily routine. Possible causes include psychological, biological, and social sources of distress. Clinical depression is the more-severe form of depression, also known as major depression or major depressive disorder. Recently, electroencephalography and speech signals have been used for early diagnosis of depression

however, they focus on moderate or severe depression. We have combined audio spectrogram and multiple frequencies of EEG signals to improve diagnostic performance. To do so, we have fused different levels of speech and EEG features to generate descriptive features and applied vision transformers and various pre-trained networks on the speech and EEG spectrum. We have conducted extensive experiments on Multimodal Open Dataset for Mental-disorder Analysis (MODMA) dataset, which showed significant improvement in performance in depression diagnosis (<bold>0.972</bold>, <bold>0.973</bold> and <bold>0.973</bold> precision, recall and F1 score respectively ) for patients at the mild stage. Besides, we provided a web-based framework using Flask and provided the source code publicly <uri>https://github.com/RespectKnowledge/EEG_Speech_Depression_MultiDL</uri>. IEEE