Transformers in medical image analysis

Kelei He, Chen Gan, Zhuoyuan Li, Islem Rekik, Zihao Yin, Wen Ji, Yang Gao, Qian Wang (Lead / Corresponding author), Junfeng Zhang (Lead / Corresponding author), Dinggang Shen (Lead / Corresponding author)

Research output: Contribution to journalReview articlepeer-review

82 Citations (Scopus)
247 Downloads (Pure)

Abstract

Transformers have dominated the field of natural language processing and have recently made an impact in the area of computer vision. In the field of medical image analysis, transformers have also been successfully used in to full-stack clinical applications, including image synthesis/reconstruction, registration, segmentation, detection, and diagnosis. This paper aimed to promote awareness of the applications of transformers in medical image analysis. Specifically, we first provided an overview of the core concepts of the attention mechanism built into transformers and other basic components. Second, we reviewed various transformer architectures tailored for medical image applications and discuss their limitations. Within this review, we investigated key challenges including the use of transformers in different learning paradigms, improving model efficiency, and coupling with other techniques. We hope this review would provide a comprehensive picture of transformers to readers with an interest in medical image analysis.

Original languageEnglish
Pages (from-to)59-78
Number of pages20
JournalIntelligent Medicine
Volume3
Issue number1
Early online date24 Aug 2022
DOIs
Publication statusPublished - 7 Feb 2023

Keywords

  • Deep learning
  • Diagnosis
  • Image synthesis
  • Medical image analysis
  • Multi-modal learning
  • Multi-task learning
  • Registration
  • Segmentation
  • Transformer
  • Weakly-supervised learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Biomedical Engineering
  • Health Informatics
  • Medicine (miscellaneous)

Fingerprint

Dive into the research topics of 'Transformers in medical image analysis'. Together they form a unique fingerprint.

Cite this