Skip to main navigation Skip to search Skip to main content

Optimising the usability of AI-assisted augmented reality displays of critical structures during surgery – An international study of surgeon-computer interaction

  • CARES Evaluation Group
  • , Roxana Ramirez Herrera
  • , Danyal Z. Khan (Lead / Corresponding author)
  • , Anjana Wijekoon
  • , Sophia Bano
  • , Matthew J. Clarkson
  • , Hani J. Marcus
  • , Ann Blandford

Research output: Contribution to journal › Article › peer-review

9 Downloads (Pure)

Abstract

In many endoscopic surgical procedures, the surgical team must identify and remove pathological tissue while avoiding critical structures such as arteries and nerves. Augmented reality (AR) offers potential support by overlaying visual information about the location of pathology and critical structures directly onto the operative field, enhancing spatial awareness and surgical navigation. However, limited research has evaluated how best to design and present AR overlays in ways that align with surgical workflow and perception. This study investigates surgeons’ preferences across three key AR overlay dimensions: Design (how anatomy is visualised: outlines, heatmaps, masks, or centroids), Trigger (how and when overlays are activated: always visible, activated by the user, or triggered by instrument position), and Placement (where the overlay appears: above or below the surgical instrument). We take endoscopic pituitary adenoma surgery as a high-risk exemplar. Using a web-based prototype, 38 neurosurgeons ranked options and provided qualitative feedback. Surgeons preferred outline designs for clarity, with a trend towards user-activated triggers for control of information flow and distraction minimisation, and below-instrument placement for better spatial awareness. Preferences were consistent across experience levels and emphasised the importance of balancing visual saliency with cognitive load, to facilitate surgical navigation without distraction or disruption. These findings inform AR interface design, but require evaluation for impact on surgical performance and safety in further physical simulation and clinical studies.

Original languageEnglish
Article number103902
Number of pages14
JournalInternational Journal of Human Computer Studies
Volume217
Early online date6 Aug 2026
DOIs
Publication statusE-pub ahead of print - 6 Aug 2026

Keywords

  • Anatomy segmentation
  • Augmented reality
  • Human-centred surgical AI
  • Pituitary adenoma surgery
  • Surgical decision support

ASJC Scopus subject areas

  • Human Factors and Ergonomics
  • Software
  • Education
  • General Engineering
  • Human-Computer Interaction
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Optimising the usability of AI-assisted augmented reality displays of critical structures during surgery – An international study of surgeon-computer interaction'. Together they form a unique fingerprint.

Cite this