Automated emotion recognition based on higher order statistics and deep learning algorithm

Abstract

The objective of this paper is online recognition of human emotions based on electroencephalogram (EEG) signals. The emotions are originated from the central and peripheral nervous systems. Hence, it can be adequately characterized by the EEG signal, as it directly reflects changes in the human emotional states. This paper describes an automated classification of emotions-labeled EEG signals using nonlinear higher order statistics and deep learning algorithm. The discrete wavelet transform is used to decompose the studied signal into sub-bands, known as rhythms of the EEG signal. The third-order cumulants (ToC) are used to explore the nonlinear dynamics of each sub-band signal in higher dimensional space. The data in the higher dimensional space contain repeated and redundant information due to presence of various symmetries in the ToC. Hence, an evolutionary data reduction technique, namely, the particle swarm optimization, is employed to get rid of irrelevant information. The long short-term memory based deep learning technique is used to retrieve the emotion variation from the optimized data corresponding to the labeled EEG signals. This study is carried out with the web-available DEAP dataset that yields 82.01% average classification accuracy with 10-fold cross-validation technique corresponding to four-labeled emotions classes. The achieved results have confirmed that the proposed algorithm has the potential for accurate and rapid recognition of human emotions. © 2020 Elsevier Ltd