Learning person-person interaction in collective activity recognition

Xiaobin Chang, Wei Shi Zheng (Lead / Corresponding author), Jianguo Zhang

Research output: Contribution to journalArticlepeer-review

32 Citations (Scopus)


Collective activity is a collection of atomic activities (individual person's activity) and can hardly be distinguished by an atomic activity in isolation. The interactions among people are important cues for recognizing collective activity. In this paper, we concentrate on modeling the person-person interactions for collective activity recognition. Rather than relying on hand-craft description of the person-person interaction, we propose a novel learning-based approach that is capable of computing the class-specific person-person interaction patterns. In particular, we model each class of collective activity by an interaction matrix, which is designed to measure the connection between any pair of atomic activities in a collective activity instance. We then formulate an interaction response (IR) model by assembling all these measurements and make the IR class specific and distinct from each other. A multitask IR is further proposed to jointly learn different person-person interaction patterns simultaneously in order to learn the relation between different person-person interactions and keep more distinct activity-specific factor for each interaction at the same time. Our model is able to exploit discriminative low-rank representation of person-person interaction. Experimental results on two challenging data sets demonstrate our proposed model is comparable with the state-of-the-art models and show that learning person-person interactions plays a critical role in collective activity recognition.

Original languageEnglish
Article number7055886
Pages (from-to)1905-1918
Number of pages14
JournalIEEE Transactions on Image Processing
Issue number6
Early online date6 Mar 2015
Publication statusPublished - Jun 2015


  • action analysis
  • collective activity recognition
  • interaction modeling

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software


Dive into the research topics of 'Learning person-person interaction in collective activity recognition'. Together they form a unique fingerprint.

Cite this