Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

Graeme McLeod (Lead / Corresponding author), Mel McKendrick, Alasdair Taylor, Joanna Lynch, Jean Ker, Amy Sadler, Jayne Halcrow, Gary McKendrick, Ayman Mustafa, Jonathan Seeley, Pavan Raju, George Corner

Research output: Contribution to journalArticle

Abstract

Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable.

Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasound-guided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses.

Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95% confidence interval [CI]: 0.78-0.89) and 0.90 (95% CI: 0.87-0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95% CI: 1.5-4.8]; P<0.001), had less errors (mean difference: 4.9 [95% CI: 3.5-6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95% CI: 3.6-10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95% CI: 0-288]; P=0.048). Fixation count correlated negatively with steps (r=-0.60; P=0.04) and with errors (r=0.64; P=0.03).

Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.

Original languageEnglish
Pages (from-to)368-377
Number of pages10
JournalBritish Journal of Anaesthesia
Volume123
Issue number3
Early online date27 Jun 2019
DOIs
Publication statusPublished - Sep 2019

Fingerprint

Conduction Anesthesia
Cadaver
Reproducibility of Results
Confidence Intervals
Checklist
Nerve Block
Psychometrics
Needles
Glass
Volunteers
Anatomy

Keywords

  • cadaver
  • eye tracking
  • interscalene block
  • metrics
  • regional anaesthesia
  • simulation
  • ultrasound guidance
  • Patient Simulation
  • Reproducibility of Results
  • Ultrasonography, Interventional/methods
  • Humans
  • Education, Medical, Graduate/methods
  • Psychometrics
  • Scotland
  • Nerve Block/standards
  • Delphi Technique
  • Translational Medical Research/methods
  • Checklist
  • Anesthesiology/education
  • Cadaver

Cite this

McLeod, Graeme ; McKendrick, Mel ; Taylor, Alasdair ; Lynch, Joanna ; Ker, Jean ; Sadler, Amy ; Halcrow, Jayne ; McKendrick, Gary ; Mustafa, Ayman ; Seeley, Jonathan ; Raju, Pavan ; Corner, George. / Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients. In: British Journal of Anaesthesia. 2019 ; Vol. 123, No. 3. pp. 368-377.
@article{c5fcaf8d6d38458491d11f073248a826,
title = "Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients",
abstract = "Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable.Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasound-guided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses.Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95{\%} confidence interval [CI]: 0.78-0.89) and 0.90 (95{\%} CI: 0.87-0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95{\%} CI: 1.5-4.8]; P<0.001), had less errors (mean difference: 4.9 [95{\%} CI: 3.5-6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95{\%} CI: 3.6-10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95{\%} CI: 0-288]; P=0.048). Fixation count correlated negatively with steps (r=-0.60; P=0.04) and with errors (r=0.64; P=0.03).Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.",
keywords = "cadaver, eye tracking, interscalene block, metrics, regional anaesthesia, simulation, ultrasound guidance, Patient Simulation, Reproducibility of Results, Ultrasonography, Interventional/methods, Humans, Education, Medical, Graduate/methods, Psychometrics, Scotland, Nerve Block/standards, Delphi Technique, Translational Medical Research/methods, Checklist, Anesthesiology/education, Cadaver",
author = "Graeme McLeod and Mel McKendrick and Alasdair Taylor and Joanna Lynch and Jean Ker and Amy Sadler and Jayne Halcrow and Gary McKendrick and Ayman Mustafa and Jonathan Seeley and Pavan Raju and George Corner",
year = "2019",
month = "9",
doi = "10.1016/j.bja.2019.04.060",
language = "English",
volume = "123",
pages = "368--377",
journal = "British Journal of Anaesthesia",
issn = "0007-0912",
publisher = "PROF SCI PUBL",
number = "3",

}

McLeod, G, McKendrick, M, Taylor, A, Lynch, J, Ker, J, Sadler, A, Halcrow, J, McKendrick, G, Mustafa, A, Seeley, J, Raju, P & Corner, G 2019, 'Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients', British Journal of Anaesthesia, vol. 123, no. 3, pp. 368-377. https://doi.org/10.1016/j.bja.2019.04.060

Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients. / McLeod, Graeme (Lead / Corresponding author); McKendrick, Mel; Taylor, Alasdair; Lynch, Joanna; Ker, Jean; Sadler, Amy; Halcrow, Jayne; McKendrick, Gary; Mustafa, Ayman; Seeley, Jonathan; Raju, Pavan; Corner, George.

In: British Journal of Anaesthesia, Vol. 123, No. 3, 09.2019, p. 368-377.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

AU - McLeod, Graeme

AU - McKendrick, Mel

AU - Taylor, Alasdair

AU - Lynch, Joanna

AU - Ker, Jean

AU - Sadler, Amy

AU - Halcrow, Jayne

AU - McKendrick, Gary

AU - Mustafa, Ayman

AU - Seeley, Jonathan

AU - Raju, Pavan

AU - Corner, George

PY - 2019/9

Y1 - 2019/9

N2 - Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable.Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasound-guided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses.Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95% confidence interval [CI]: 0.78-0.89) and 0.90 (95% CI: 0.87-0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95% CI: 1.5-4.8]; P<0.001), had less errors (mean difference: 4.9 [95% CI: 3.5-6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95% CI: 3.6-10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95% CI: 0-288]; P=0.048). Fixation count correlated negatively with steps (r=-0.60; P=0.04) and with errors (r=0.64; P=0.03).Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.

AB - Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable.Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasound-guided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses.Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95% confidence interval [CI]: 0.78-0.89) and 0.90 (95% CI: 0.87-0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95% CI: 1.5-4.8]; P<0.001), had less errors (mean difference: 4.9 [95% CI: 3.5-6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95% CI: 3.6-10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95% CI: 0-288]; P=0.048). Fixation count correlated negatively with steps (r=-0.60; P=0.04) and with errors (r=0.64; P=0.03).Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.

KW - cadaver

KW - eye tracking

KW - interscalene block

KW - metrics

KW - regional anaesthesia

KW - simulation

KW - ultrasound guidance

KW - Patient Simulation

KW - Reproducibility of Results

KW - Ultrasonography, Interventional/methods

KW - Humans

KW - Education, Medical, Graduate/methods

KW - Psychometrics

KW - Scotland

KW - Nerve Block/standards

KW - Delphi Technique

KW - Translational Medical Research/methods

KW - Checklist

KW - Anesthesiology/education

KW - Cadaver

UR - http://www.scopus.com/inward/record.url?scp=85067892974&partnerID=8YFLogxK

U2 - 10.1016/j.bja.2019.04.060

DO - 10.1016/j.bja.2019.04.060

M3 - Article

C2 - 31255289

VL - 123

SP - 368

EP - 377

JO - British Journal of Anaesthesia

JF - British Journal of Anaesthesia

SN - 0007-0912

IS - 3

ER -