The Psychometric Confound: A Perennial Gremlin of the Psychopathology Literature

Chapman and Chapman (1973) identified an issue in psychopathology research that has since come to be known as the Psychometric Confound (MacDonald, 2008). They claimed, essentially, that various traditional inferential methods for drawing conclusions regarding ability deficits in a population with some particular pathology were flawed. The work of the Chapmans has since been cited frequently in the psychopathology field, with most citing authors echoing their concerns, and some applying their proposed solutions. However, the precise nature of the phenomenon remains in question. The goal of the current work is to elucidate, in mathematics, the issues raised by Chapman and Chapman, and their commentators, to a level which allows for an adjudication of the core claims of these authors. We begin by providing a clear and concise description of Chapman and Chapman’s account of the Psychometric Confound, including a description of the research context; an articulation of the general inferential problem; an itemization of claims, including claims regarding methodological solutions; and a description of problems inherent in Chapman and Chapman’s account. We then consider the influence of the Chapmans’ discussion regarding the Psychometric Confound on the psychopathology literature as a whole, including a summary of the alternative accounts of the problem that have emerged in response to the work of Chapman and Chapman. A full mathematization, and consequent adjudication, of the claims of Chapman and Chapman, is then provided. Fundamentally, this involves an elucidation and formalization of the test theory, both classical and modern, nascent in all work regarding the Psychometric Confound from Chapman and Chapman on. A mathematization and adjudication of the claims of the alternative accounts follows. Finally, we determine if valid methodological solutions for the quantities of interest are possible, given the technical, test-theory based framework established. We show that a structural equation model consistent with the proto-framework implied by Chapman and Chapman provides a basis for valid inference regarding the quantity of interest.