Computerized adaptive testing of population psychological distress

simulation-based evaluation of GHQ-30

Jan Stochl (Lead / Corresponding author), Jan Boehnke, Kate E. Pickett, Tim J. Croudace (Lead / Corresponding author)

Research output: Contribution to journalArticle

4 Citations (Scopus)
79 Downloads (Pure)

Abstract

Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting.

Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria.

Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations.
Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.
Original languageEnglish
Pages (from-to)895-906
Number of pages12
JournalSocial Psychiatry and Psychiatric Epidemiology
Volume51
Issue number6
Early online date21 Dec 2015
DOIs
Publication statusPublished - Jun 2016

Fingerprint

Psychology
simulation
questionnaire
Health
health
evaluation
Population
Surveys and Questionnaires
cross-sectional study
longitudinal study
Health Surveys
Longitudinal Studies
Life Style
efficiency
Cross-Sectional Studies

Keywords

  • Computerized adaptive testing
  • Item response theory
  • Bifactor model
  • Measurement invariance
  • GENERAL HEALTH QUESTIONNAIRE

Cite this

@article{28d9e981dc534afcb8c3abd805853e68,
title = "Computerized adaptive testing of population psychological distress: simulation-based evaluation of GHQ-30",
abstract = "Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting.Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria.Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations.Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.",
keywords = "Computerized adaptive testing, Item response theory, Bifactor model, Measurement invariance, GENERAL HEALTH QUESTIONNAIRE",
author = "Jan Stochl and Jan Boehnke and Pickett, {Kate E.} and Croudace, {Tim J.}",
note = "TJC reports grants from GL Assessment (2008–2011) held whilst at the University of Cambridge (with Prof J Rust) for an ability test standardization project (BAS 3) outside the present work. TJC and JS report a personal fee from GL Assessment for psychometric calibration of the BAS3 (ability tests) outside the submitted work. GL Assessment market and distribute the General Health Questionnaire.",
year = "2016",
month = "6",
doi = "10.1007/s00127-015-1157-4",
language = "English",
volume = "51",
pages = "895--906",
journal = "Social Psychiatry and Psychiatric Epidemiology",
issn = "0933-7954",
publisher = "Springer Verlag",
number = "6",

}

Computerized adaptive testing of population psychological distress : simulation-based evaluation of GHQ-30. / Stochl, Jan (Lead / Corresponding author); Boehnke, Jan; Pickett, Kate E.; Croudace, Tim J. (Lead / Corresponding author).

In: Social Psychiatry and Psychiatric Epidemiology, Vol. 51, No. 6, 06.2016, p. 895-906.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Computerized adaptive testing of population psychological distress

T2 - simulation-based evaluation of GHQ-30

AU - Stochl, Jan

AU - Boehnke, Jan

AU - Pickett, Kate E.

AU - Croudace, Tim J.

N1 - TJC reports grants from GL Assessment (2008–2011) held whilst at the University of Cambridge (with Prof J Rust) for an ability test standardization project (BAS 3) outside the present work. TJC and JS report a personal fee from GL Assessment for psychometric calibration of the BAS3 (ability tests) outside the submitted work. GL Assessment market and distribute the General Health Questionnaire.

PY - 2016/6

Y1 - 2016/6

N2 - Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting.Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria.Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations.Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.

AB - Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting.Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria.Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations.Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.

KW - Computerized adaptive testing

KW - Item response theory

KW - Bifactor model

KW - Measurement invariance

KW - GENERAL HEALTH QUESTIONNAIRE

U2 - 10.1007/s00127-015-1157-4

DO - 10.1007/s00127-015-1157-4

M3 - Article

VL - 51

SP - 895

EP - 906

JO - Social Psychiatry and Psychiatric Epidemiology

JF - Social Psychiatry and Psychiatric Epidemiology

SN - 0933-7954

IS - 6

ER -