Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos

Yijun Yan, Jinchang Ren (Lead / Corresponding author), Huimin Zhao, Jiangbin Zheng, Ezrinda Mohd Zaihidee, John Soraghan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.

Original languageEnglish
Title of host publicationAdvances in Multimedia Information Processing
Subtitle of host publication17th Pacific-Rim Conference on Multimedia, PCM 2016, Proceedings
EditorsEnqing Chen, Yun Tie, Yihong Gong
Place of PublicationSwitzerland
PublisherSpringer
Pages697-704
Number of pages8
Edition1
ISBN (Electronic)978-3-319-48896-7
ISBN (Print)9783319488950
DOIs
Publication statusPublished - 2016
Event17th Pacific-Rim Conference on Multimedia, PCM 2016 - Xi’an, China
Duration: 15 Sept 201616 Sept 2016

Publication series

NameLecture Notes in Computer Science
Volume9917
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th Pacific-Rim Conference on Multimedia, PCM 2016
Country/TerritoryChina
CityXi’an
Period15/09/1616/09/16

Keywords

  • Image fusion
  • Pedestrian detection/tracking
  • Thermal image
  • Video salient objects
  • Visible image

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos'. Together they form a unique fingerprint.

Cite this