PERSIST: Improving micro-expression spotting using better feature encodings and multi-scale Gaussian TCN

Abstract

Micro-expression (ME) is required in real-world applications for understanding true human feeling. The preliminary step of ME analysis, ME spotting, is highly challenging for human experts because MEs induce subtle facial movements for a short duration. Moreover, the existing feature encodings are insufficient for spotting because they are affected by illumination and eye-blinking. These issues are alleviated for better ME spotting by our proposed method, PERSIST, that is, imProved fEatuRe encodingS and multIscale gauSsian Temporal convolutional network. It investigates the possibility of human gaze deformations for spotting. In contrast to the well-known sequence models like RNN and LSTM, it explores the feasibility of a temporal convolutional network to model long-term dependencies in a better way. Furthermore, the proposed network efficacy is significantly improved by adding a Gaussian filter layer and performing multi-resolution analysis. Experimental results conducted on publicly available ME spotting databases reveal that our method PERSIST outperforms the well-known methods. It also indicates that eyebrow information is helpful in ME spotting when eye-blinking artifacts are mitigated, and human gaze information can be consolidated with other encodings for performance improvement. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.