Towards probabilistic representation learning of continuous-time dynamics

This dissertation studies the problem of learning continuous-time dynamics from discrete observations in probabilistic approaches. We first propose an normalizing flow model for continuous-time stochastic processes that transform simple Wiener processes into more complex processes using indexed invertible mappings. Such designs permit efficient sampling and exact density evaluation given arbitrary time grids on the transformed processes. We further propose to augment the indexed normalizing flows with an expressive generic neural stochastic differential equations to model latent dynamics. Indexed normalizing flows will decode trajectories of the latent dynamics into continuous processes in the observation space. A piece-wise approach to the latent and variational posterior processes is proposed for optimization and inference. The thesis concludes with an extension of the popular particle filtering method to continuous-time models based on latent neural stochastic differential equations using the same piece-wise treatment and importance weighting. The proposed particle filtering method can serve as a better drop-in replacement for inference methods based on variational approximations and shows consistent improvements across models and tasks.