Egomap: Hierarchical First-person Semantic Mapping

Tamas Suveges, Stephen McKenna (Lead / Corresponding author)

Research output: Chapter in Book/Report/Conference proceedingConference contribution

88 Downloads (Pure)


We consider unsupervised learning of semantic, user-specific maps from first-person video. The task we address can be thought of as a semantic, non-geometric form of simultaneous localisation and mapping, differing in significant ways from formulations typical in robotics. Locations, termed stations, typically correspond to rooms or areas in which a user spends time, places to which they might refer in spoken conversation. Our maps are modeled as a hierarchy of probabilistic station graphs and view graphs. View graphs capture an aspect of user behaviour within stations. Visits are temporally segmented based on qualitative visual motion and used to update the map, either by updating an existing map station or adding a new map station. We contribute a labelled dataset suitable for evaluation of this novel SLAM task. Experiments compare mapping performance with and without the use of view graphs and demonstrate better online mapping than when using offline clustering.
Original languageEnglish
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges
Subtitle of host publicationVirtual Event, January 10–15, 2021, Proceedings, Part III
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
Place of PublicationSwitzerland
Number of pages16
ISBN (Electronic)9783030687960
ISBN (Print)9783030687953
Publication statusPublished - 2021
Event2nd Workshop on Applications of Egocentric Vision (EgoApp): In conjunction with ICPR 2020, Milan, Italy - Milan, Italy
Duration: 10 Jan 202115 Jan 2021

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference2nd Workshop on Applications of Egocentric Vision (EgoApp)
Internet address

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Egomap: Hierarchical First-person Semantic Mapping'. Together they form a unique fingerprint.

Cite this