Improving healthcare policies using reinforcement learning on patterns of service utilization

Reinforcement Learning (RL) is an important class of methods in Artificial Intelligence (AI), particularly for optimization problems and decision-making under uncertainty. However, practical and ethical concerns in healthcare settings can limit the application of traditional RL methods, requiring innovative approaches. This thesis explores the application of RL methods in healthcare to evaluate treatment strategies. We begin with an overview of RL, followed by an introduction to Q-Learning and Dyna-Q, two fundamental RL algorithms. We demonstrate the application of these algorithms using a simulated robot, AdventureBot, navigating a grid world. We then introduce Hidden Markov Mixture Models (HMMMs) as a method for extracting patient subgroups with distinct patterns from longitudinal data, which we apply to a simulated dataset. Finally, we describe our proposed pipeline for integrating HMMMs with CFRL to evaluate healthcare policies in an offline setting.