An evaluation of the reliability and quality of expert and novice forensic case formulations

Forensic case formulation is an under-studied and growing area within the violence risk assessment literature. The current study aimed to address gaps in the literature by examining the interrater reliability (IRR) and quality of forensic case formulations by comparing Expert and Novice raters. N = 50 intimate partner violence offender files were accessed. Four raters (n = 2 Experts, n = 2 Novices) rated each file using all steps of Spousal Assault Risk Assessment Guide-Version 3 (SARA-V3; Kropp & Hart, 2015). Cases were formulated using a Decision Theory approach in which motivating, disinhibiting, and destabilizing mechanisms were identified. The distribution of ratings for these mechanisms was presented. IRR was examined using a novel coefficient, Gwet’s AC. Raters also completed narrative case formulations. Then a Within Case and Across Case paired case design involving n = 143 narrative formulation pairs was conducted with three new raters. The similarity of paired formulations was evaluated. Raters also assessed the quality of formulations using the Case Formulation Quality Checklist-Revised (CFQC-R; McMurran & Bruford, 2016). For most formulation mechanisms, distribution of Presence ratings was skewed. Overall, across Experts and Novices, the IRR of formulation mechanisms ranged from poor to almost perfect (AC2 = .10. - .98), with most coefficients falling between the moderate and almost perfect ranges. The similarity of formulations was established; Within Case paired formulations were judged as more similar than Across Case paired formulations. Finally, formulations were high in quality; Experts produced higher quality formulations than Novices.