Intelligent Tutoring Systems and Learning Outcomes: Two Systematic Reviews

Intelligent Tutoring Systems (ITSs) are computer programs that dynamically model learners’ psychological states to provide individualized instruction. ITSs have been developed for diverse subjects to help learners to acquire domain-specific, cognitive and metacognitive knowledge at all educational levels. In this thesis, I report on two studies conducted to examine the current state of the ITS field. The first study is a meta-analysis conducted on research that compared the outcomes from students learning from ITSs to those learning from non-ITS learning environments. It examines 107 studies, published prior to 2013, with a total of 14,321 participants. The results show that ITSs outperform teacher-led, large-group instruction (g = .42), non-ITS computer-based instruction (g = .57), and textbooks or workbooks (g = .35). However, no statistically significant difference was detected between learning from ITS and learning from individualized human tutoring (g = -.11) or small-group instruction (g = .05). The second study evaluates research on the relative effectiveness of Bayesian networks in constructing student models in ITSs, which involves 143 studies published between 1992 and 2014. The study explores how Bayesian network was adopted to support the development of student models, relative to its strengths and weaknesses in investigating learning constructs and their contributions to the effectiveness of BN student modeling. A number of implications are drawn with respect to the application of BN in ITS design. Both reviews provide evidence that ITSs are relatively effective tools for learning. Furthermore, ITS researchers are invited to reconsider three fundamental research questions that have been examined since the emergence of ITSs and how they contribute to and constrain advances in effective ITS design in light of developments in artificial intelligence research. Finally, recommendations for future research directions are provided to researchers in the ITS community.