Visual Tracking and Region Alignment in Surgical and Medical Education

To design a user computer interactive system for medical and surgical education, a study of visual tracking and region alignment is presented in this thesis. To effectively track the surgical instrument in the view of an endoscope, both the Gaussian type tracking methods, based on the Extended Kalman Filter (EKF) and the Adaptive Gaussian Mixture Model (AGMM) and the non-Gaussian type Particle Filter (PF) approach are proposed and evaluated using the video data captured from both in-vitro and in-vivo environments. To improve the stability of instrument tracking, a hybrid method integrating PF and AGMM is presented. One of the extensions of the visual tracking results is to match the same region under different viewing conditions and combine it with Augmented Reality (AR). A preliminary study of user-defined region matching algorithms, based on feature descriptors such as SIFT, SURF and ORB, are proposed and integrated with AR development.