Sensemaking with learning analytics visualizations: Investigating dashboard comprehension and effects on learning strategy

In the provision of just-in-time feedback, student-facing learning analytics dashboards (LADs) are meant to aid decision-making during the process of learning. Unlike summative feedback received at its conclusion, this formative feedback may help learners pivot their learning strategies while still engaged in the learning activity. To turn this feedback into actionable insights however, learners must understand LADs well enough to make accurate judgements of learning with them. For these learners, LADs could become an integral part of their self-regulatory learning strategy. This dissertation presents a multifaceted examination of learners' sensemaking processes with LADs designed to support self-regulatory learning. The in-situ studies detailed therein examine learners' understanding of the data visualized in LADs and the effects of this understanding on their performance-related mental models. Trace data, surveys, semi-structured in-depth qualitative interviews, and retrospective cued recall methods were used to identify why, when, and how learners used LADs to guide their learning. Learners' qualitative accounts of their experience explained and contextualized the quantitative data collected from the observed activities. Learners preferred less complex LADs, finding them more useful and aesthetically appealing, despite lower gist recall with simpler visualizations. During an early investigation of how LADs were used to make learning judgments in situ, we observed learners' tendency to act upon brief LAD interactions. This inspired us to operationalize gist as a form of measurement, describing learners' ability to make sense of a LAD after a brief visual interrogation. Subsequent comparisons of the accuracy and descriptiveness of learners' gist estimates to those of laypeople repeatedly showed that laypeople were more apt than learners to produce accurate and complete gist descriptions. This dissertation culminates in a final study examining the evolution of learners' mental models of their performance due to repeated LAD interaction, followed by a discussion of the contextual factors that contributed to what was observed. Trends observed across this work suggest that learners were more apt to "get the gist" with LAD after repeated interaction. This dissertation contributes a novel method for evaluating learners' interpretation of LADs, while our findings offer insight into how LADs shape learners' sensemaking processes.