Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video

Sophia Bano; Jianguo Zhang; Stephen McKenna

doi:10.1109/ICCVW.2017.274

Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video

Sophia Bano, Jianguo Zhang, Stephen McKenna

Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Citations (Scopus)

271 Downloads (Pure)

Abstract

Focused interaction occurs when co-present individuals, having mutual focus of attention, interact by establishing face-to-face engagement and direct conversation. Face-toface engagement is often not maintained throughout the entirety of a focused interaction. In this paper, we present an online method for automatic classification of unconstrained egocentric (first-person perspective) videos into segments having no focused interaction, focused interaction when the camera wearer is stationary and focused interaction when the camera wearer is moving. We extract features from both audio and video data streams and perform temporal segmentation by using support vector machines with linear and non-linear kernels. We provide empirical evidence that fusion of visual face track scores, camera motion profile and audio voice activity scores is an effective combination for focused interaction classification.

Original language	English
Title of host publication	2017 IEEE International Conference on Computer Vision Workshop (ICCVW)
Publisher	IEEE
Pages	2322-2330
Number of pages	9
ISBN (Electronic)	9781538610343
ISBN (Print)	9781538610350
DOIs	https://doi.org/10.1109/ICCVW.2017.274
Publication status	Published - 23 Jan 2018
Event	IEEE International Conference on Computer Vision Workshops - Venice Convention Centre, Venice, Italy Duration: 22 Oct 2017 → 29 Oct 2017 http://iccv2017.thecvf.com/

Publication series

Name	Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017
Volume	2018-January

Conference

Conference	IEEE International Conference on Computer Vision Workshops
Abbreviated title	ICCV 2017
Country/Territory	Italy
City	Venice
Period	22/10/17 → 29/10/17
Internet address	http://iccv2017.thecvf.com/

Keywords

Cameras
Face
Feature extraction
Tracking
Visualization
Legged locomotion

ASJC Scopus subject areas

Computer Vision and Pattern Recognition
Computer Science Applications

Access to Document

10.1109/ICCVW.2017.274

Author Accepted ManuscriptAccepted author manuscript, 5.96 MB

Cite this

Bano, S., Zhang, J., & McKenna, S. (2018). Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video. In 2017 IEEE International Conference on Computer Vision Workshop (ICCVW) (pp. 2322-2330). (Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017; Vol. 2018-January). IEEE. https://doi.org/10.1109/ICCVW.2017.274

@inproceedings{b418ae4e50cc49e7b09325362aa8ec7a,

title = "Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video",

abstract = "Focused interaction occurs when co-present individuals, having mutual focus of attention, interact by establishing face-to-face engagement and direct conversation. Face-toface engagement is often not maintained throughout the entirety of a focused interaction. In this paper, we present an online method for automatic classification of unconstrained egocentric (first-person perspective) videos into segments having no focused interaction, focused interaction when the camera wearer is stationary and focused interaction when the camera wearer is moving. We extract features from both audio and video data streams and perform temporal segmentation by using support vector machines with linear and non-linear kernels. We provide empirical evidence that fusion of visual face track scores, camera motion profile and audio voice activity scores is an effective combination for focused interaction classification.",

keywords = "Cameras, Face, Feature extraction, Tracking, Visualization, Legged locomotion",

author = "Sophia Bano and Jianguo Zhang and Stephen McKenna",

note = "This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant EP/N014278/1: ACE-LP: Augmenting Communication using Environmental Data to drive Language Prediction. The authors are grateful to Annalu Waller (University of Dundee), the ACE-LP team and CVIP members (University of Dundee) for useful discussions and assistance with dataset collection.; IEEE International Conference on Computer Vision Workshops, ICCV 2017 ; Conference date: 22-10-2017 Through 29-10-2017",

year = "2018",

month = jan,

day = "23",

doi = "10.1109/ICCVW.2017.274",

language = "English",

isbn = "9781538610350",

series = "Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017",

publisher = "IEEE",

pages = "2322--2330",

booktitle = "2017 IEEE International Conference on Computer Vision Workshop (ICCVW)",

url = "http://iccv2017.thecvf.com/",

}

Bano, S, Zhang, J & McKenna, S 2018, Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video. in 2017 IEEE International Conference on Computer Vision Workshop (ICCVW). Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017, vol. 2018-January, IEEE, pp. 2322-2330, IEEE International Conference on Computer Vision Workshops, Venice, Italy, 22/10/17. https://doi.org/10.1109/ICCVW.2017.274

Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video. / Bano, Sophia; Zhang, Jianguo ; McKenna, Stephen.
2017 IEEE International Conference on Computer Vision Workshop (ICCVW). IEEE, 2018. p. 2322-2330 (Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Finding Time Together

T2 - IEEE International Conference on Computer Vision Workshops

AU - Bano, Sophia

AU - Zhang, Jianguo

AU - McKenna, Stephen

N1 - This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant EP/N014278/1: ACE-LP: Augmenting Communication using Environmental Data to drive Language Prediction. The authors are grateful to Annalu Waller (University of Dundee), the ACE-LP team and CVIP members (University of Dundee) for useful discussions and assistance with dataset collection.

PY - 2018/1/23

Y1 - 2018/1/23

N2 - Focused interaction occurs when co-present individuals, having mutual focus of attention, interact by establishing face-to-face engagement and direct conversation. Face-toface engagement is often not maintained throughout the entirety of a focused interaction. In this paper, we present an online method for automatic classification of unconstrained egocentric (first-person perspective) videos into segments having no focused interaction, focused interaction when the camera wearer is stationary and focused interaction when the camera wearer is moving. We extract features from both audio and video data streams and perform temporal segmentation by using support vector machines with linear and non-linear kernels. We provide empirical evidence that fusion of visual face track scores, camera motion profile and audio voice activity scores is an effective combination for focused interaction classification.

AB - Focused interaction occurs when co-present individuals, having mutual focus of attention, interact by establishing face-to-face engagement and direct conversation. Face-toface engagement is often not maintained throughout the entirety of a focused interaction. In this paper, we present an online method for automatic classification of unconstrained egocentric (first-person perspective) videos into segments having no focused interaction, focused interaction when the camera wearer is stationary and focused interaction when the camera wearer is moving. We extract features from both audio and video data streams and perform temporal segmentation by using support vector machines with linear and non-linear kernels. We provide empirical evidence that fusion of visual face track scores, camera motion profile and audio voice activity scores is an effective combination for focused interaction classification.

KW - Cameras

KW - Face

KW - Feature extraction

KW - Tracking

KW - Visualization

KW - Legged locomotion

UR - http://www.scopus.com/inward/record.url?scp=85046168242&partnerID=8YFLogxK

U2 - 10.1109/ICCVW.2017.274

DO - 10.1109/ICCVW.2017.274

M3 - Conference contribution

SN - 9781538610350

T3 - Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017

SP - 2322

EP - 2330

BT - 2017 IEEE International Conference on Computer Vision Workshop (ICCVW)

PB - IEEE

Y2 - 22 October 2017 through 29 October 2017

ER -

Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

ACE-LP - Augmenting Communication Using Environmental Data to Drive Language Prediction (Joint with University of Cambridge)

Multimodal Focused Interaction Dataset

Cite this

Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Projects

ACE-LP - Augmenting Communication Using Environmental Data to Drive Language Prediction (Joint with University of Cambridge)

Datasets

Multimodal Focused Interaction Dataset

Cite this