Discovery - University of Dundee - Online Publications

Library & Learning Centre

Jointly learning heterogeneous features for RGB-D activity recognition

Jointly learning heterogeneous features for RGB-D activity recognition

Research output: Contribution to journalArticle

View graph of relations


  • Jian-Fang Hu
  • Wei-Shi Zheng (Lead / Corresponding author)
  • Jianhuang Lai
  • Jianguo Zhang

Research units


Original languageEnglish
Pages (from-to)5344-5352
Number of pages9
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number11
Early online date15 Dec 2016
StatePublished - Nov 2017
EventThe IEEE Conference on Computer Vision and Pattern Recognition - Boston, United States


In this paper, we focus on heterogeneous feature learning for RGB-D activity recognition. Considering that features from different channels could share some similar hidden structures, we propose a joint learning model to simultaneously explore the shared and feature-specific components as an instance of heterogenous multi-task learning. The proposed model in an unified framework is capable of: 1) jointly mining a set of subspaces with the same dimensionality to enable the multi-task classifier learning, and 2) meanwhile, quantifying the shared and feature-specific components of features in the subspaces. To efficiently train the joint model, a three-step iterative optimization algorithm is proposed, followed by two inference models. Extensive results on three activity datasets have demonstrated the efficacy of the proposed method. In addition, a novel RGB-D activity dataset focusing on human-object interaction is collected for evaluating the proposed method, which will be made available to the community for RGB-D activity benchmarking and analysis

Download statistics

No data available


Open Access permissions



  • Author Accepted Manuscript

    Accepted author manuscript, 7 MB, PDF-document

    Copyright 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.



Library & Learning Centre

Contact | Accessibility | Policy