Action recognition in videos using deep learning approaches

Abstract

Given the inherent complexity of video data, action recognition in videos poses a formidable challenge in computer vision. The 3D space-time volume encompassing frame sequences contains substantial redundant information, diverting the model from acquiring a discriminative representation of the performed action class. Although 3D Convolutional Neural Networks (3D CNNs) exhibit exceptional spatio-temporal feature learning capabilities, leading to state-of-the-art action recognition performance on various large-scale benchmark video datasets, a naive 3D CNN architecture comes with drawbacks. Firstly, it demonstrates incompetence in modeling long-range dependencies due to the fixed and limited receptive field of the 3D convolutional kernel. Secondly, its demand for a substantial amount of data and extensive computational time during training arises from the number of parameters involved. Recently, much research has focused on alleviating the limitation of 3D CNNs. Various techniques have tried to increase the 3D CNN model’s depth by stacking multiple convolutional layers. Although expanding the depth has compensated for the 3D kernel’s limited receptive field, it has exploded the model’s parameter, making its need for training data and computation time consumption critical.