Impact of Video Compression on Object Tracking Performance

Video compression is ubiquitous in virtually all visual processing pipelines nowadays. However, its impact on many visual analytics tasks is not fully understood. In this thesis, we study the impact of video compression on multiple object tracking (MOT). MOT problem is essentially the assignment of unique identifiers to the multiple objects. In our case, the multiple objects are initialized through object detection. To achieve multiple object tracking, we employed detection-based tracking, which requires a detector to detect the objects before the identifier (ID) assignments. We used You Only Look Once v3 (YOLOv3) for object detection and Simple Online Realtime Tracking (SORT) to perform identities association to the detected objects. We collected the data by applying High Efficiency Video Coding (HEVC) to uncompressed video sequences, varying quantization parameter (QP) and motion search range (MSR). We then ran the object tracker composed of YOLOv3 and SORT on decoded sequences. We focused our study on Multiple Object Tracking Accuracy (MOTA) and found that QP has significant impact on the MOTA score, but MSR does not. We also proposed the relationship between MOTA and QP. Analyzing the results from compressed and uncompressed video sequences, we understood that most performance metrics decrease as QP increases and the image quality drops. Increase of MOTA and Precision were observed in some video sequences because in certain cases, YOLOv3 detects wrong objects. We also found that occlusion makes a difference in ID switch (IDs). IDs decreases in the video with frequent occlusions as QP increases but increases in the video with fewer occlusions. Finally, the impact of detection on tracking performance was studied. MOTA and Mean Average Precision at Intersection over Union (IOU) threshold of 0.5 (mAP-50) are positively correlated, and we observed that the relationship between MOTA and mAP-50 is linear in some video sequences. However, we also observed non-linear cases such that the MOTA growth rate decreases as mAP-50 increases in some video sequences.