Context Aware Text Prediction for Enhancing Augmentative Communication

Description

People without natural speech such as Lost Voice Guy, winner of BGT, use keyboard-based applications to communicate. When an additional physical impairment makes it impossible to type fast, for example due to Motor Neuron Disease, it becomes a challenge to generate sufficiently fast spoken text. Timely and effective access to good predictions of words and whole sentences becomes crucial.

We will showcase a new alternative system which uses AI computer vision, analysing a live video stream of what the person using the communication system sees. The system identifies context information such as the type of location and the communication partner and presents word and sentence predictions directly to the person’s fingertips.

The system will be presented by the principal investigator who herself has a speech impairment and successfully uses the system to speed up her typing on a touch screen.

Our research draws from the knowledge that the bulk of human communication is based on personal narrative – sharing experiences and personal stories more than once with a person’s wider social circle. This text generating support has applications far beyond communication devices and broadens application beyond the field of assistive technology.