Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation

Gerry T. M. Altmann, Yuki Kamide

    Research output: Contribution to journalArticlepeer-review

    110 Citations (Scopus)

    Abstract

    Two experiments explored the mapping between language and mental representations of visual scenes. In both experiments, participants viewed, for example, a scene depicting a woman, a wine glass and bottle on the floor, an empty table, and various other objects. In Experiment 1, participants concurrently heard either 'The woman will put the glass on the table' or 'The woman is too lazy to put the glass on the table'. Subsequently, with the scene unchanged, participants heard that the woman 'will pick up the bottle, and pour the wine carefully into the glass.' Experiment 2 was identical except that the scene was removed before the onset of the spoken language. In both cases, eye movements after 'pour' (anticipating the glass) and at 'glass' reflected the language-determined position of the glass, as either on the floor, or moved onto the table, even though the concurrent (Experiment I) or prior (Experiment 2) scene showed the glass in its unmoved position on the floor. Language-mediated eye movements thus reflect the real-time mapping of language onto dynamically updateable event-based representations of concurrently or previously seen objects (and their locations). (C) 2008 Elsevier B.V. All rights reserved.

    Original languageEnglish
    Pages (from-to)55-71
    Number of pages17
    JournalCognition
    Volume111
    Issue number1
    DOIs
    Publication statusPublished - Apr 2009

    Cite this