Learning to recognise behaviours of persons with dementia using multiple cues in an HMM-based approach

Christian Peters, Sven Wachsmuth, Jesse Hoey

    Research output: Chapter in Book/Report/Conference proceedingOther chapter contribution

    11 Citations (Scopus)


    This paper presents a learning technique for visual event recognition in a system that assists persons with dementia during handwashing. The challenge is that persons with dementia present a wide variety of behaviors during a single task, typically changing their behaviours drastically from day to day. Any attempt at modeling this variety requires a large set of features, image regions, and temporal dynamics. In this paper, we approach this challenge by supervised learning of generative models from manually segmented and labelled video sequences. Our method uses a generic set of appearance-based colour, motion and texture features, over a static set of regions. We then present two HMM architectures that incorporate multiple image regions by either fusing on a feature-level, or later in the recognition process using a mixture-of-experts approach, in which a gating HMM is applied for the dynamic selection between specialised expert HMMs. Our models are trained on a clinical database of videos, and we compare the HMM approaches with a nearest neighbours scheme. Our results confirm the challenge we present, and indicate that our generative modelling techniques are suitable for inclusion in future prototypes of the hand washing assistant.
    Original languageEnglish
    Title of host publicationPETRA '09
    Subtitle of host publicationProceedings of the 2nd International Conference on PErvasive Technologies Related to Assistive Environments
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery
    ISBN (Print)978-1-60558-409-6
    Publication statusPublished - 2009


    Dive into the research topics of 'Learning to recognise behaviours of persons with dementia using multiple cues in an HMM-based approach'. Together they form a unique fingerprint.

    Cite this