Skip to Main content Skip to Navigation
Journal articles

Hybrid dual stream blender for wide baseline view synthesis

Abstract : Free navigation of a scene requires warping some reference views to some desired target viewpoint and blending them to synthesize a virtual view. Convolutional Neural Networks (ConvNets) based methods can learn both the warping and blending tasks jointly. Such methods are often designed for moderate inter-camera baseline distance and larger kernels are required for warping if the baseline distance increases. Algorithmic methods can in principle deal with large baselines, however the synthesized view suffers from artifacts near disoccluded pixels. We present a hybrid approach where first, reference views are algorithmically warped to the target position and then are blended via a ConvNet. Preliminary view warping allows reducing the size of the convolutional kernels and thus the learnable parameters count. We propose a residual encoder-decoder for image blending with a Siamese encoder to further keep the parameters count low. We also contribute a hole inpainting algorithm to fill the disocclusions in the warped views. Our view synthesis experiments on real multiview sequences show better objective image quality than state-of-the-art methods due to fewer artifacts in the synthesized images.
Document type :
Journal articles
Complete list of metadata
Contributor : Laurent Jonchère Connect in order to contact the contributor
Submitted on : Monday, September 6, 2021 - 4:29:38 PM
Last modification on : Tuesday, October 19, 2021 - 10:36:38 PM


 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2021-12-30

Please log in to resquest access to the document



Nour Hobloss, Lu Zhang, Stephane Lathuiliere, Marco Cagnazzo, Attilio Fiandrotti. Hybrid dual stream blender for wide baseline view synthesis. Signal Processing: Image Communication, Elsevier, 2021, 97, pp.116366. ⟨10.1016/j.image.2021.116366⟩. ⟨hal-03330264⟩



Record views