Date of Graduation
Fall 2021
Degree
Master of Natural and Applied Science in Computer Science
Department
Computer Science
Committee Chair
Razib Iqbal
Abstract
Missing information in the video frames is estimated as close as possible to the actual data during video error concealment process. Blocks or slices of information in the video frames can be missing in the decoder due to various reasons like corrupt media drives, network congestion, etc. which reduces the quality of experience for the viewers. One approach to deal with missing information in the video decoder is to use error concealment techniques to fill the missing information. Until now many of these error concealment techniques were based on conventional methods such as block copy, motion vector prediction, and interpolation. In this thesis, I propose Convergent Error Concealment Neural Network (CECNN) for video error concealment. It is based on the Convolutional Neural Network (CNN) architecture that takes an errored frame as input and gives an error concealed frame as output. I applied transfer learning to produce intermediate outputs and combine them to fill the missing information in the errored frames. Unlike commonly used single path neural network architecture, my proposed CECNN consists of separate paths for the preceding and succeeding frames to produce the intermediate outputs. It also supports a single path using the preceding frames for time-sensitive use cases, such as real-time video. Experimental results confirm that my proposed network gives better error concealed output for real-time video communication. Quality metrics like MS-SSIM, PSNR, and MSE are used to calculate and compare the quality of my proposed technique’s output with the output obtained from motion vector estimation, neighboring motion vectors, and neural network-based generative image inpainting. The experimental results show that my CECNN approach is a good candidate to replace the existing conventional techniques to error conceal the errored frames in any video decoder.
Keywords
error concealment, video codec, convolutional neural network, transfer learning, quality metrics
Subject Categories
Computer Sciences
Copyright
© Shashi Khanal
Recommended Citation
Khanal, Shashi, "Video Error Concealment Using Convolutional Neural Network" (2021). MSU Graduate Theses/Dissertations. 3689.
https://bearworks.missouristate.edu/theses/3689
