Enhancing skeleton-based action recognition using a knowledge-driven shift graph convolutional network

Abstract

Recently, there has been a noticeable upsurge in the exploration of human action recognition (HAR) using skeleton data. Compared with video data, skeleton data has advantages, such as lightweightness and high resilience to changes in appearance, lighting conditions, and camera angles. Graph convolutional networks (GCNs) excel in feature extraction from skeletal data, which is non-Euclidean by nature. However, many existing GCN-based methods suffer from high complexities and rigid receptive fields. Consequently, attention has shifted toward the development of lighter architectures with fewer parameters. One such approach is the Shift-GCN, which integrates lightweight feature shift operations to enhance the adaptability of spatiotemporal receptive fields. Despite efficiently capturing distant spatial relationships, this method struggles to differentiate similar actions with subtle variations and requires additional graph connection information. To this end, this study proposes a knowledge-driven shift-GCN (KDS-GCN). The proposed model generates a more detailed and nuanced feature representation by leveraging the integration of graph connectivity knowledge with coordinate information, combined with a lightweight and flexible shift framework, thereby improving action recognition performance. Experiments on four benchmark datasets exhibit the superior performance of the proposed KDS-GCN model while demanding lower computational resources than existing methods. © 2024 Elsevier Ltd