Real-time 3D tracking of laparoscopy training instruments for assessment and feedback

Benjamin Gautier, Harun Tugal, Benjie Tang, Ghulam Nabi, Mustafa Suphi Erden (Lead / Corresponding author)

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)
59 Downloads (Pure)


Assessment of minimally invasive surgical skills is a non-trivial task, usually requiring the presence and time of expert observers, including subjectivity and requiring special and expensive equipment and software. Although there are virtual simulators that provide self-assessment features, they are limited as the trainee loses the immediate feedback from realistic physical interaction. The physical training boxes, on the other hand, preserve the immediate physical feedback, but lack the automated self-assessment facilities. This study develops an algorithm for real-time tracking of laparoscopy instruments in the video cues of a standard physical laparoscopy training box with a single fisheye camera. The developed visual tracking algorithm recovers the 3D positions of the laparoscopic instrument tips, to which simple colored tapes (markers) are attached. With such system, the extracted instrument trajectories can be digitally processed, and automated self-assessment feedback can be provided. In this way, both the physical interaction feedback would be preserved and the need for the observance of an expert would be overcome. Real-time instrument tracking with a suitable assessment criterion would constitute a significant step towards provision of real-time (immediate) feedback to correct trainee actions and show them how the action should be performed. This study is a step towards achieving this with a low cost, automated, and widely applicable laparoscopy training and assessment system using a standard physical training box equipped with a fisheye camera.
Original languageEnglish
Article number751741
Number of pages11
JournalFrontiers in Robotics and AI
Publication statusPublished - 4 Nov 2021


  • training
  • real-time motion tracking
  • Cartesian position estimation
  • single view camera
  • skill assessment
  • skill metric
  • cartesian position estimation
  • laparoscopy training
  • laparacospy

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications


Dive into the research topics of 'Real-time 3D tracking of laparoscopy training instruments for assessment and feedback'. Together they form a unique fingerprint.

Cite this