Date of Graduation

Fall 2021

Degree

Master of Natural and Applied Science in Computer Science

Department

Computer Science

Committee Chair

Razib Iqbal

Keywords

error concealment, video codec, convolutional neural network, transfer learning, quality metrics

Subject Categories

Computer Sciences

Abstract

Missing information in the video frames is estimated as close as possible to the actual data during video error concealment process. Blocks or slices of information in the video frames can be missing in the decoder due to various reasons like corrupt media drives, network congestion, etc. which reduces the quality of experience for the viewers. One approach to deal with missing information in the video decoder is to use error concealment techniques to fill the missing information. Until now many of these error concealment techniques were based on conventional methods such as block copy, motion vector prediction, and interpolation. In this thesis, I propose Convergent Error Concealment Neural Network (CECNN) for video error concealment. It is based on the Convolutional Neural Network (CNN) architecture that takes an errored frame as input and gives an error concealed frame as output. I applied transfer learning to produce intermediate outputs and combine them to fill the missing information in the errored frames. Unlike commonly used single path neural network architecture, my proposed CECNN consists of separate paths for the preceding and succeeding frames to produce the intermediate outputs. It also supports a single path using the preceding frames for time-sensitive use cases, such as real-time video. Experimental results confirm that my proposed network gives better error concealed output for real-time video communication. Quality metrics like MS-SSIM, PSNR, and MSE are used to calculate and compare the quality of my proposed technique’s output with the output obtained from motion vector estimation, neighboring motion vectors, and neural network-based generative image inpainting. The experimental results show that my CECNN approach is a good candidate to replace the existing conventional techniques to error conceal the errored frames in any video decoder.

Copyright

© Shashi Khanal

Available for download on Friday, December 01, 2023

Open Access

Share

COinS