In which fields are citations indicators of research quality?

Mike Thelwall, Kayvan Kousha, Emma Stuart, Meiko Makita, Mahshid Abdoli, Paul Wilson, Jonathan Levitt

Research output: Contribution to journalArticlepeer-review

22 Citations (Scopus)
114 Downloads (Pure)

Abstract

Citation counts are widely used as indicators of research quality to support or replace human peer review and for lists of top cited papers, researchers, and institutions. Nevertheless, the relationship between citations and research quality is poorly evidenced. We report the first large-scale science-wide academic evaluation of the relationship between research quality and citations (field normalized citation counts), correlating them for 87,739 journal articles in 34 field-based UK Units of Assessment (UoA). The two correlate positively in all academic fields, from very weak (0.1) to strong (0.5), reflecting broadly linear relationships in all fields. We give the first evidence that the correlations are positive even across the arts and humanities. The patterns are similar for the field classification schemes of Scopus and Dimensions.ai, although varying for some individual subjects and therefore more uncertain for these. We also show for the first time that no field has a citation threshold beyond which all articles are excellent quality, so lists of top cited articles are not pure collections of excellence, and neither is any top citation percentile indicator. Thus, while appropriately field normalized citations associate positively with research quality in all fields, they never perfectly reflect it, even at high values.
Original languageEnglish
Pages (from-to)941-953
Number of pages13
JournalJournal of the Association for Information Science and Technology
Volume74
Issue number8
Early online date4 May 2023
DOIs
Publication statusPublished - Aug 2023

Fingerprint

Dive into the research topics of 'In which fields are citations indicators of research quality?'. Together they form a unique fingerprint.

Cite this