Disocclusion hole-filling in DIBR-synthesized images using multi-scale template matching

Smarti Reel, K. C. P. Wong, Gene Cheung, Laurence S. Dooley

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Transmitting texture and depth images of captured camera view(s) of a 3D scene enables a receiver to synthesize novel virtual viewpoint images via Depth-Image-Based Rendering (DIBR). However, a DIBR-synthesized image often contains disocclusion holes, which are spatial regions in the virtual view image that were occluded by foreground objects in the captured camera view(s). In this paper, we propose to complete these disocclusion holes by exploiting the self-similarity characteristic of natural images via nonlocal template-matching (TM). Specifically, we first define self-similarity as nonlocal recurrences of pixel patches within the same image across different scales-one characterization of self-similarity in a given image is the scale range in which these patch recurrences take place. Then, at encoder we segment an image into multiple depth layers using available per-pixel depth values, and characterize self-similarity in each layer with a scale range; scale ranges for all layers are transmitted as side information to the decoder. At decoder, disocclusion holes are completed via TM on a per-layer basis by searching for similar patches within the designated scale range. Experimental results show that our method improves the quality of rendered images over previous disocclusion hole-filling algorithms by up to 3.9dB in PSNR.

Original languageEnglish
Title of host publication2014 IEEE Visual Communications and Image Processing Conference, VCIP 2014
PublisherIEEE
Pages494-497
Number of pages4
ISBN (Electronic)9781479961399
DOIs
Publication statusPublished - 2 Mar 2015
Event2014 IEEE Visual Communications and Image Processing Conference, VCIP 2014 - Valletta, Malta
Duration: 7 Dec 201410 Dec 2014

Conference

Conference2014 IEEE Visual Communications and Image Processing Conference, VCIP 2014
CountryMalta
CityValletta
Period7/12/1410/12/14

Keywords

  • depth-image-based rendering
  • Free viewpoint video
  • image inpainting

Fingerprint Dive into the research topics of 'Disocclusion hole-filling in DIBR-synthesized images using multi-scale template matching'. Together they form a unique fingerprint.

  • Cite this

    Reel, S., Wong, K. C. P., Cheung, G., & Dooley, L. S. (2015). Disocclusion hole-filling in DIBR-synthesized images using multi-scale template matching. In 2014 IEEE Visual Communications and Image Processing Conference, VCIP 2014 (pp. 494-497). [7051614] IEEE. https://doi.org/10.1109/VCIP.2014.7051614