Tampere University of Technology

TUTCRIS Research Portal

Tensor-based linear dynamical systems for action recognition from 3D skeletons

Research output: Contribution to journalArticleScientificpeer-review


Original languageEnglish
Pages (from-to)75-86
Number of pages12
JournalPattern Recognition
Early online date5 Dec 2017
Publication statusPublished - May 2018
Publication typeA1 Journal article-refereed


Abstract Recent years have seen a growth in interest in skeleton-based human behavior recognition. Skeleton sequences can be expressed naturally as high-order tensor time series, and in this paper we report on the modeling and analysis of such time series using a linear dynamical system (LDS). Owing to their relative simplicity and efficiency, LDSs are the most common tool used in various disciplines for encoding spatiotemporal time series data. However, conventional LDSs process the latent and observed states at each frame of a video as a column vector, a representation that fails to take into account valuable structural information associated with human action. To correct this, we propose a tensor-based linear dynamical system (tLDS) for modeling tensor observations in time series and employ Tucker decomposition to estimate the parameters of the LDS model as action descriptors. In this manner, an action can be expressed as a subspace corresponding to a point on a Grassmann manifold on which classification can be performed using dictionary learning and sparse coding. Experiments using the MSR Action3D, UCF Kinect, and Northwestern-UCLA Multiview Action3D datasets demonstrate the excellent performance of our proposed method.


  • Skeleton joints, Action recognition, Subspace learning, Tensor learning, Grassmann manifold

Publication forum classification

Field of science, Statistics Finland