Probabilistic solution of differential equations for Bayesian uncertainty quantification and inference

In many areas of applied science the time and space evolution of variables can be naturally described by differential equation models, which define states implicitly as functions of their own rates of change. Inference for differential equation models requires an explicit representation of the states (the solution), which is typically not known in closed form, but can be approximated by a variety of discretization-based numerical methods. However, numerical error analysis is not well-suited for describing functional discretization error in a way that can be propagated through the inverse problem, and is consequently ignored in practice. Because its impact can be substantial, characterizing the effect of discretization uncertainty propagation on inference has been an important open problem. We develop a probability model for the systematic uncertainty introduced by a finite-dimensional representation of the infinite-dimensional solution of ordinary and partial differential equation problems. The result is a probability distribution over the space of possible state trajectories, describing our belief about the unknown solution given information generated from the model over a discrete grid. Our probabilistic approach provides a useful alternative to deterministic numerical integration techniques in cases when models are chaotic, ill-conditioned, or contain unmodelled functional variability. Based on these results, we develop a fully probabilistic Bayesian approach for the statistical inverse problem of inference and prediction for intractable differential equation models from data, which characterizes and propagates discretization uncertainty in the estimation. Our approach is demonstrated on a number of challenging forward and inverse problems.