Improving the performance of bundle adjustment for on-device SLAM using GPU resources

Visual-inertial SLAM systems estimate the state and trajectory of a moving device while building a map of the environment using sensors such as cameras and inertial measurement units. These systems apply a general form of bundle adjustment to reduce error accumulated in the relationships between keyframes, map points, and inertial states. We present techniques to accelerate bundle adjustment for on-device SLAM using GPU resources. First, we develop Vulkan compute shaders for calculating the Schur complement of a sparse matrix to accelerate local visual-inertial bundle adjustment. Next, we extend this work for larger-scale global bundle adjustment problems by developing an iterative linear solver for explicit and implicit approaches. To evaluate the performance, we integrate our methods into a graph optimization library, g2o, and visual-inertial SLAM system, ORB-SLAM3, and process a mix of indoor and outdoor datasets on desktop and embedded devices. We also test our methods on large-scale bundle adjustment datasets.