Validity and feasibility study of direct observation of radiotherapy planning skills

Purpose or Objective Radiotherapy planning skill (RPS) is a fundamental skill for radiation oncologists. There is a dearth of research investigating the use of Direct Observation of Procedural skills (DOPS) for RPS. There are variations in the DOPS tools among different radiation oncology training programs internationally. The aim of this study was to examine the validity and feasibility of DOPS for RPS in the Irish post graduate training program.
Material and Methods This is a quantitative study aligned with the positivism paradigm. Data were collected using an online survey. All the trainers and trainees in the Irish postgraduate radiation oncology training program were invited to complete a survey questionnaire. Total population sample was used. To examine content validity, a potential checklist of 34 items was developed and responders were asked to rate each step according to its importance. Questionnaire included other questions related to the face validity and feasibility like the DOPS scoring system and frequency of need. The acquired data were analysed by descriptive statistics. Both Content validity ratio (CVR) and Content validity index (CVI) were calculated.
Results Twenty three out of invited 47 trainers and trainees completed the survey questionnaire (13 trainers and 10 trainees). The items CVI and CVR for a potential 34 items checklist were calculated. An Updated checklist containing 16 items which met the validity threshold was created and achieved excellent validity (S-CVI and CVR 0.93 and 0.87 respectively). A minimum of 4 DOPS of RPS per year is considered feasible. There is an agreement to keep the scoring system for each step based the expected performance of the trainees according to their stage of training. However, there is no agreement on the global score. The majority recommended development of site specific DOPS tools.
Conclusion This study showed the feasibility of DOPS. It illustrated acceptable quantity indices of content validity for a 16 item checklist for DOPS of RPS. Based on the study results, I recommend implementing an updated DOPS checklist with a minimum of 4 DOPS of RPS per year as formative assessment in the Irish radiation oncology postgraduate training program.