25 / 2023-08-28 11:20:39

Multi-View Contrastive Self-supervised Triplet network for Skeleton-based Action Recognition

self-supervised learning, skeleton-based action recognition, contrastive learning, multi-view modeling, triplet network

Special Sessions > SS02 - Intelligent Robot, Artificial Intelligence Methods and Its Applications

Improving feature discrimination power from different viewpoints has long been a hot topic of interest for skeleton-based action recognition. Most of the existing methods are based on fully supervised approaches for feature representation learning, which need to rely on a large number of labeled samples. However, it is easy to lead to overfitting and insufficient generalization of the model. In this work, we propose a novel Multi-View contrastive self-supervised Triplet network for skeleton-based action recognition, dubbed MVT. The original skeleton is encoded with features from different viewpoints, and mutual information maximization through a triplet network combined with triplet-view loss by contrastive learning. In the finetune stage of the downstream task, we froze the encoder combined with a classifier to output the final classification score. The extensive experiments show that our proposed MVT dramatically outperforms the state-of-the-art self-supervised learning methods for NTU-60, NTU-120 and PKU-MMD benchmarks.