Computerized adaptive testing of population psychological distress: simulation-based evaluation of GHQ-30

TY - JOUR

T1 - Computerized adaptive testing of population psychological distress

T2 - simulation-based evaluation of GHQ-30

AU - Stochl, Jan

AU - Boehnke, Jan

AU - Pickett, Kate E.

AU - Croudace, Tim J.

N1 - TJC reports grants from GL Assessment (2008–2011) held whilst at the University of Cambridge (with Prof J Rust) for an ability test standardization project (BAS 3) outside the present work. TJC and JS report a personal fee from GL Assessment for psychometric calibration of the BAS3 (ability tests) outside the submitted work. GL Assessment market and distribute the General Health Questionnaire.

PY - 2016/6

Y1 - 2016/6

N2 - Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting. Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria. Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations. Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.

AB - Purpose: Goldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting. Methods: GHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria. Results: To yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations. Conclusions: The rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.

KW - Computerized adaptive testing

KW - Item response theory

KW - Bifactor model

KW - Measurement invariance

KW - GENERAL HEALTH QUESTIONNAIRE

U2 - 10.1007/s00127-015-1157-4

DO - 10.1007/s00127-015-1157-4

M3 - Article

C2 - 26687370

SN - 0933-7954

VL - 51

SP - 895

EP - 906

JO - Social Psychiatry and Psychiatric Epidemiology

JF - Social Psychiatry and Psychiatric Epidemiology

IS - 6

ER -