Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype

Pavithran Maniam (Lead / Corresponding author), Philipp Schnell, Lilly Dan, Rony Portelli, Caroline Erolin, Rodney Mountain, Tracey Wilkinson

Research output: Contribution to journalArticle

Abstract

Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.

Original languageEnglish
Pages (from-to)17-26
Number of pages10
JournalJournal of Visual Communication in Medicine
Volume43
Issue number1
Early online date24 Oct 2019
DOIs
Publication statusPublished - 2020

Fingerprint

Temporal Bone
Anatomy
Teaching
Technology
Anatomic Models
Equipment and Supplies
Medical Education
Head
Prototype
Mixed Reality
Resources
Learning
Students
Bone and Bones
Operator

Keywords

  • Temporal bone
  • surgical
  • HoloLens
  • mixed reality
  • ENT
  • educational tool

Cite this

@article{7412a7cfa2234dbabb09699288a32d82,
title = "Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype",
abstract = "Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.",
keywords = "Temporal bone, surgical, HoloLens, mixed reality, ENT, educational tool",
author = "Pavithran Maniam and Philipp Schnell and Lilly Dan and Rony Portelli and Caroline Erolin and Rodney Mountain and Tracey Wilkinson",
note = "Funding - Medtronic Public Limited Company (to P.M., P.S., R.P. and L.D.) to develop a mixed reality educational tool using HoloLens.",
year = "2020",
doi = "10.1080/17453054.2019.1671813",
language = "English",
volume = "43",
pages = "17--26",
journal = "Journal of Visual Communication in Medicine",
issn = "1745-3054",
publisher = "Taylor & Francis",
number = "1",

}

Exploration of temporal bone anatomy using mixed reality (HoloLens) : development of a mixed reality anatomy teaching resource prototype. / Maniam, Pavithran (Lead / Corresponding author); Schnell, Philipp; Dan, Lilly; Portelli, Rony; Erolin, Caroline; Mountain, Rodney; Wilkinson, Tracey.

In: Journal of Visual Communication in Medicine, Vol. 43, No. 1, 2020, p. 17-26.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Exploration of temporal bone anatomy using mixed reality (HoloLens)

T2 - development of a mixed reality anatomy teaching resource prototype

AU - Maniam, Pavithran

AU - Schnell, Philipp

AU - Dan, Lilly

AU - Portelli, Rony

AU - Erolin, Caroline

AU - Mountain, Rodney

AU - Wilkinson, Tracey

N1 - Funding - Medtronic Public Limited Company (to P.M., P.S., R.P. and L.D.) to develop a mixed reality educational tool using HoloLens.

PY - 2020

Y1 - 2020

N2 - Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.

AB - Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.

KW - Temporal bone

KW - surgical

KW - HoloLens

KW - mixed reality

KW - ENT

KW - educational tool

UR - http://www.scopus.com/inward/record.url?scp=85076463528&partnerID=8YFLogxK

U2 - 10.1080/17453054.2019.1671813

DO - 10.1080/17453054.2019.1671813

M3 - Article

C2 - 31645155

VL - 43

SP - 17

EP - 26

JO - Journal of Visual Communication in Medicine

JF - Journal of Visual Communication in Medicine

SN - 1745-3054

IS - 1

ER -