Presentation Matters: A Design Study of Different Keyboard Layouts to Investigate the Use of Prediction for AAC

Background: Augmentative and Alternative Communication (AAC) applications on mobile technology typically use on-screen keyboards for text entry. AAC users with limited dexterity tend to use single finger typing, leading to slow text entry rates. Although word and phrase prediction have the potential to increase rates, users tend to keep typing rather than selecting predictions. It is hypothesized that the need to scan lists of predicted words or phrases necessitates an undesired shift of gaze, resulting in missed predictions.

Method: We propose new ways for on-screen presentation of predicted words and phrases and report on early results from a multiple single user study using three different keyboard layouts. Keyboard layout designs were informed by a literature review, an online questionnaire and focus group activities with participants who use AAC. Three on-screen keyboards were implemented on a touchscreen tablet – a standard keyboard and two layouts which display predictions closer to where the user’s visual attention is already focused. The Standard Layout (SL) displays a row of 4 pre- S142 Part 5: Communication dicted words above the keyboard; the ‘Above Typed Layout’ (TL) displays up to four predicted words in a 2 × 2 grid above the typed letter; while the ‘Above Predicted Layout’ (PL) displays up to 4 predicted words above the next predicted letters. Ten participants with neurological impairments affecting hand function were invited to copy type a number of short memorable phrases. All participants used single finger typing apart from one participant who used eye gaze with mouse pointer control for access. Due to physical limitations we were only able to record the gaze of four participants during their typing exercise using an eye gaze tracker. A semi structured interview was conducted after the activity. For the four participants with gaze data, we analysed text entry rate, error rate and personal preference using videos of the typing interaction, screen captures with gaze plots and verbal feedback.

Key results: Participants expressed different preferences of keyboards after the experiment. These ranged from preference for the standard layout (SL) due to the familiarity, to preference for PL due to perceived accommodation of finger and gaze movement. Entry rate was the highest using SL. Participants achieved the highest keystroke savings with PL and significantly lower savings with TL. Participants missed predictions using all layouts. Interview feedback conflicted with observed entry rates (e.g. preferring a slower keyboard layout, perceiving this to have a faster entry rate).

Conclusion: Preliminary results suggest that displaying predictions closer to the next letter to be typed may increase the selection of predictions. Although text entry rates were higher using SL, this may change with longer use of PL. Further studies which include the integration of phrase prediction into the PL keyboard and extended use are being undertaken