Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable.

Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasound-guided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses.

Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95% confidence interval [CI]: 0.78-0.89) and 0.90 (95% CI: 0.87-0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95% CI: 1.5-4.8]; P<0.001), had less errors (mean difference: 4.9 [95% CI: 3.5-6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95% CI: 3.6-10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95% CI: 0-288]; P=0.048). Fixation count correlated negatively with steps (r=-0.60; P=0.04) and with errors (r=0.64; P=0.03).

Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.