Multilevel investigation of variation in HoNOS ratings by mental health professionals: a naturalistic study of consecutive referrals

Episodes of mental healthcare in specialist psychiatric services often begin with the assessment of clinical and psychosocial needs of patients by healthcare professionals. Particularly for patients with complex needs or severe problems, ratings of clinical and social functioning at the start of each episode of care may serve as a baseline against which subsequent measures can be compared. Currently, little is known about service variations in such assessments on referrals from primary care. We set out to quantify variability in initial assessments performed by healthcare professionals in three CMHTs in Bristol (UK) using the Health of the Nation Outcome Scales (HoNOS). We tested the hypothesis that variations in HoNOS total and sub-scale scores are related to referral source (general practices), healthcare assessor (in CMHTs) and the assessor's professional group. Statistical analysis was performed using multilevel variance components models with cross-classified random effects. We found that variation due to assessor substantially exceeded that due to referral source (general practices). Furthermore, patient variance differed by assessor profession for the HoNOS--Impairment scores. Assessor variance differed by assessor profession for the HoNOS--Social scores. As HoNOS total and subscale scores show much larger variation by assessor than by referral source, investigations of HoNOS scores must take assessors into account. Services should implement and evaluate interdisciplinary training to improve consistency in use of rating thresholds; such initiatives could be evaluated using these extensions of multilevel models. Future research should aim to integrate routine diagnostic data with continuous outcomes to address selection effects (of patients to assessors) better.