Use of digital records for studying skill learning

The present work uses a novel data source, real-time strategy video game play in StarCraft 2, to study complex skill learning. Chapter One discusses some important desiderata of a large dataset. Chapter Two discusses domain specifics about StarCraft 2, and introduces the process by which survey respondents donate digital archives which are parsed to reveal second-by-second information about in-game performance of players. Chapter Three asks how experience should be defined in a complex domain. I find that the common-sense definition, that experience should be measured soley in terms ot task-specific experience, misleads researchers by being both overly permissive and restrictive. A better definition can be achieved by focusing on other forms of experience, such as experience with different game modes. Chapter Four extends a previous study of age-related declines in a StarCraft 2 cross-sectional dataset. Segmented regression models are used to estimate the onset of age-related differences. Secondly, I examine the theory that large swaths of age-related differences, across a wide array of variables, are attributable to a single general cognitive, but not psychomotor, factor. I find support for this theory, as a simplified measure of redundant click-speed accounts for about 19\% of the shared age-related variance in established measures of StarCraft 2 speed. In Chapter Five I examine some of the common responses to the idea that Big Data, and the emerging data sources they employ, could effectively replace the role of theory in science. I argue, instead, that emerging data sources are a threat to overzealous generalizations from laboratory grown theories to complex behaviour. If emerging data sources fulfill their potential as tools for evaluating theory generality, then scientific standards for making claims about generality could change in pronounced ways. This would create a bigger gap between empirically grounded generalizations from the laboratory to life and careless generalizations which Frankfurt would call ``bullshit.'' Finally, I examine two very different research strategies for going about the evaluation of theory using Big Data, and point to the virtues and limitations of both.