Visual-Inertial Sensor Fusion for Tracking in Ambulatory Environments

Tracking a high-velocity object through a cluttered environment is daunting for even the human-observer. Vision-based trackers will frequently lose their lock on the object as features on the object become distorted or faded as a result of motion-blurring imparted by the high-velocity of the object. Moreover, the frequent occlusions as the object passes through clutter only serves to compound the issue. To boot, the usual difficulties associated with most vision-based trackers still apply such as: nonuniform illumination, object rotation, object scale changes, etc... Inertial-based trackers provide useful complementary data to aid the vision-based systems. The higher sampling rates of the inertial measurements gives invaluable information to be able to track high-speed objects. With the IMU attached to the object, the inertial measurements are immune to occlusions unlike their visual counterparts. Efficient combination of visual as well as inertial sensors into a unified framework is coined visual-inertial sensor fusion. Visual-inertial sensor fusion is a powerful tool for many industries: it allows the medical practitioners to better understand and diagnose illnesses; it allows the engineer to design more flexible and immersive virtual reality environments; and it allows the film-director to fully capture motion in a scene. The complementary nature of visual and inertial sensors is well-toted throughout these industries, the faster sampling rate of the inertial sensors fits lock-and-key with the higher accuracy of the visual sensor to unlock the potential for algorithms capable of tracking high-velocity objects through cluttered environments. Inevitably, sensor fusion is accompanied by higher algorithmic complexity and requires careful understanding of the components involved. For this reason, the approach taken in this thesis is a ground-up approach towards a complete visual-inertial system: from camera calibration all the way to handling of asynchronous sensor measurements for sensor-fusion.