Measuring self-regulatory phases with multi-channel trace data in open-ended learning technology

Research has emphasized that self-regulated learning (SRL) is critically important for learning. Students have different capabilities of regulating their learning processes and individual needs. To help students improve their SRL capabilities, we need to identify students' current behaviors. Specifically, I used learning design to create visible and meaningful markers of student progress through SRL in an open-ended technology, a Learning Management System (LMS). I applied knowledge engineering to develop a framework of proximal indicators representing SRL phases and evaluated them in quasi-experiments in four different learning activities. I developed an embedded tool to collect real-time students' self-reports in the LMS. Comparing two SRL measures, i.e., behavioral and self-reported measures, revealed a relatively high agreement between two measures (weighted kappa, κ = .62 - .74). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting phases, compared with students' real-time self-reported behaviors. Our behavioral indicators also were comparably successful at classifying SRL phases for different students' cohorts. The revised indicators incorporating SRL temporal features improved the convergence of behavioral and self-reported measures. However, the findings revealed that the SRL temporal feature is task-specific and requires customization for a specific task type. The findings also suggested that the task type may influence how students progress through SRL processes. Overall, this thesis demonstrated how the triangulation of multiple sources of students' self-regulatory data could help unravel the complex nature of SRL phases. Additionally, this thesis highlighted the importance of learning design in open-ended learning technology to support students and track their progress through SRL phases for personalized scaffolds.