Implementation Fidelity and Attainment in Computerized Practice of Mathematics

Keith Topping (Lead / Corresponding author)

Research output: Contribution to journalArticle

Abstract

Measuring the implementation fidelity (IF) or integrity of interventions is crucial, otherwise a positive or negative outcome cannot be interpreted. Direct and indirect methods of IF measurement tend to over-emphasize teacher behaviour. This paper focuses on IF measured by student behaviour collected through computers. Attainment was measured by the STAR test of maths (a computerized item-banked adaptive norm-referenced test). Implementation quality (IF) was measured by Accelerated Maths (AM) (an instruction-free personalized practice and progress-monitoring system in mastery of mathematics skills). Attainment data was gathered in the UK on 20,103 students in 148 schools, and of these implementation data on n=6,285. Only a small percentage of pupils scored on five AM implementation indices at or above the levels recommended. Correlations between attainment and implementation indices were modest, but high implementation was positively correlated with high attainment. Socio-economic status did not appear to affect implementation or attainment. Implementation quality of AM is clearly a problem in the UK, and needs improvement. However, overall students still scored above average on attainment.
Original languageEnglish
Number of pages19
JournalResearch Papers in Education
Early online date11 Apr 2019
DOIs
Publication statusE-pub ahead of print - 11 Apr 2019

Fingerprint

mathematics
teachers' behavior
student
integrity
pupil
monitoring
instruction
school
economics

Keywords

  • Mathematics
  • attainment
  • computerized assessment
  • implementation
  • implementation fidelity
  • implementation integrity
  • improving classroom teaching
  • students

Cite this

@article{6d93657b4de845649bf478c1cf42cdc6,
title = "Implementation Fidelity and Attainment in Computerized Practice of Mathematics",
abstract = "Measuring the implementation fidelity (IF) or integrity of interventions is crucial, otherwise a positive or negative outcome cannot be interpreted. Direct and indirect methods of IF measurement tend to over-emphasize teacher behaviour. This paper focuses on IF measured by student behaviour collected through computers. Attainment was measured by the STAR test of maths (a computerized item-banked adaptive norm-referenced test). Implementation quality (IF) was measured by Accelerated Maths (AM) (an instruction-free personalized practice and progress-monitoring system in mastery of mathematics skills). Attainment data was gathered in the UK on 20,103 students in 148 schools, and of these implementation data on n=6,285. Only a small percentage of pupils scored on five AM implementation indices at or above the levels recommended. Correlations between attainment and implementation indices were modest, but high implementation was positively correlated with high attainment. Socio-economic status did not appear to affect implementation or attainment. Implementation quality of AM is clearly a problem in the UK, and needs improvement. However, overall students still scored above average on attainment.",
keywords = "Mathematics, attainment, computerized assessment, implementation, implementation fidelity, implementation integrity, improving classroom teaching, students",
author = "Keith Topping",
year = "2019",
month = "4",
day = "11",
doi = "10.1080/02671522.2019.1601759",
language = "English",
journal = "Research Papers in Education",
issn = "0267-1522",
publisher = "Taylor & Francis",

}

TY - JOUR

T1 - Implementation Fidelity and Attainment in Computerized Practice of Mathematics

AU - Topping, Keith

PY - 2019/4/11

Y1 - 2019/4/11

N2 - Measuring the implementation fidelity (IF) or integrity of interventions is crucial, otherwise a positive or negative outcome cannot be interpreted. Direct and indirect methods of IF measurement tend to over-emphasize teacher behaviour. This paper focuses on IF measured by student behaviour collected through computers. Attainment was measured by the STAR test of maths (a computerized item-banked adaptive norm-referenced test). Implementation quality (IF) was measured by Accelerated Maths (AM) (an instruction-free personalized practice and progress-monitoring system in mastery of mathematics skills). Attainment data was gathered in the UK on 20,103 students in 148 schools, and of these implementation data on n=6,285. Only a small percentage of pupils scored on five AM implementation indices at or above the levels recommended. Correlations between attainment and implementation indices were modest, but high implementation was positively correlated with high attainment. Socio-economic status did not appear to affect implementation or attainment. Implementation quality of AM is clearly a problem in the UK, and needs improvement. However, overall students still scored above average on attainment.

AB - Measuring the implementation fidelity (IF) or integrity of interventions is crucial, otherwise a positive or negative outcome cannot be interpreted. Direct and indirect methods of IF measurement tend to over-emphasize teacher behaviour. This paper focuses on IF measured by student behaviour collected through computers. Attainment was measured by the STAR test of maths (a computerized item-banked adaptive norm-referenced test). Implementation quality (IF) was measured by Accelerated Maths (AM) (an instruction-free personalized practice and progress-monitoring system in mastery of mathematics skills). Attainment data was gathered in the UK on 20,103 students in 148 schools, and of these implementation data on n=6,285. Only a small percentage of pupils scored on five AM implementation indices at or above the levels recommended. Correlations between attainment and implementation indices were modest, but high implementation was positively correlated with high attainment. Socio-economic status did not appear to affect implementation or attainment. Implementation quality of AM is clearly a problem in the UK, and needs improvement. However, overall students still scored above average on attainment.

KW - Mathematics

KW - attainment

KW - computerized assessment

KW - implementation

KW - implementation fidelity

KW - implementation integrity

KW - improving classroom teaching

KW - students

UR - http://www.scopus.com/inward/record.url?scp=85064488570&partnerID=8YFLogxK

U2 - 10.1080/02671522.2019.1601759

DO - 10.1080/02671522.2019.1601759

M3 - Article

JO - Research Papers in Education

JF - Research Papers in Education

SN - 0267-1522

ER -