Distortion Estimation and Graph-based Transform for Visual Communications

In this thesis, we study several visual communications problems, including joint source-channel coding for single view video transmission, transmission distortion estimation for multiview video coding, and depth video coding for multiview video applications. The first contribution in this thesis is the design and implementation of an error-resilient video conferencing system. We first develop an algorithm to estimate the decoder-side distortion in the presence of packet loss. We then design a family of very short systematic forward error correction (FEC) codes to recover lost packets. Finally, FEC codes are dynamically optimized to minimize the distortion from packet loss. The proposed scheme is demonstrated on a real-time embedded video conferencing system. A similar joint source channel coding framework can also be applied to multiview video coding applications such as free-viewpoint TV. Therefore an algorithm is needed for the encoder to estimate the distortion of the synthesized virtual view. We first derive a graphical model to analyze how random errors in the reference depth image affect the synthesized virtual view. We then consider the case where packet loss occurs in both the encoded texture and depth images during transmission, and develop a recursive algorithm to calculate the pixel level texture and depth probability distributions in the reference views. The recursive algorithm is then integrated with the graphical model method to estimate the distortion in the synthesized view. The graph-based transform has been extensively used for depth image coding in multiview video applications. In this thesis, we aim to develop a single graph-based transform for a class of depth signals. We first propose a 2-D first-order autoregression (2-D AR1) model and a 2-D graph to analyze depth signals with deterministic discontinuities. We show that the inverse of the biased Laplacian matrix of the proposed 2-D graph is exactly the covariance matrix of the proposed 2-D AR1 model. Therefore the optimal transform are the eigenvectors of the proposed graph Laplacian. Next, we show that similar results hold when the locations of the discontinuities are randomly distributed within a confined region. The theory in this thesis can be used to design both pre-computed and signal-dependent transforms.