A cognitive forcing tool to mitigate cognitive bias: A randomised control trial

Eoin D. O'Sullivan (Lead / Corresponding author), Susie J. Schofield

Research output: Contribution to journalArticle

58 Downloads (Pure)

Abstract

Background: Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.

Methods: We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of "think aloud" interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.

Results: Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95% CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.

Conclusions: The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.

Original languageEnglish
Article number12
Pages (from-to)1-8
Number of pages8
JournalBMC Medical Education
Volume19
Issue number1
DOIs
Publication statusPublished - 8 Jan 2019

Fingerprint

trend
diagnostic
Ireland
medical student
republic
Group
physician
decision making
Teaching
interview
evidence
experience

Keywords

  • Clinical error
  • Cognitive bias
  • Decision making
  • Heuristics

Cite this

@article{e422bf588b914331b8921e17e27bea70,
title = "A cognitive forcing tool to mitigate cognitive bias: A randomised control trial",
abstract = "Background: Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.Methods: We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of {"}think aloud{"} interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.Results: Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95{\%} CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.Conclusions: The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.",
keywords = "Clinical error, Cognitive bias, Decision making, Heuristics",
author = "O'Sullivan, {Eoin D.} and Schofield, {Susie J.}",
year = "2019",
month = "1",
day = "8",
doi = "10.1186/s12909-018-1444-3",
language = "English",
volume = "19",
pages = "1--8",
journal = "BMC Medical Education",
issn = "1472-6920",
publisher = "Springer Verlag",
number = "1",

}

A cognitive forcing tool to mitigate cognitive bias : A randomised control trial . / O'Sullivan, Eoin D. (Lead / Corresponding author); Schofield, Susie J.

In: BMC Medical Education, Vol. 19, No. 1, 12, 08.01.2019, p. 1-8.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A cognitive forcing tool to mitigate cognitive bias

T2 - A randomised control trial

AU - O'Sullivan, Eoin D.

AU - Schofield, Susie J.

PY - 2019/1/8

Y1 - 2019/1/8

N2 - Background: Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.Methods: We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of "think aloud" interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.Results: Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95% CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.Conclusions: The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.

AB - Background: Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.Methods: We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of "think aloud" interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.Results: Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95% CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.Conclusions: The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.

KW - Clinical error

KW - Cognitive bias

KW - Decision making

KW - Heuristics

UR - http://www.scopus.com/inward/record.url?scp=85059797193&partnerID=8YFLogxK

U2 - 10.1186/s12909-018-1444-3

DO - 10.1186/s12909-018-1444-3

M3 - Article

C2 - 30621679

AN - SCOPUS:85059797193

VL - 19

SP - 1

EP - 8

JO - BMC Medical Education

JF - BMC Medical Education

SN - 1472-6920

IS - 1

M1 - 12

ER -