Hybrid dual stream blender for wide baseline view synthesis - Hypermedia Domain, IRT b<>com Accéder directement au contenu
Article Dans Une Revue Signal Processing: Image Communication Année : 2021

Hybrid dual stream blender for wide baseline view synthesis

Résumé

Free navigation of a scene requires warping some reference views to some desired target viewpoint and blending them to synthesize a virtual view. Convolutional Neural Networks (ConvNets) based methods can learn both the warping and blending tasks jointly. Such methods are often designed for moderate inter-camera baseline distance and larger kernels are required for warping if the baseline distance increases. Algorithmic methods can in principle deal with large baselines, however the synthesized view suffers from artifacts near disoccluded pixels. We present a hybrid approach where first, reference views are algorithmically warped to the target position and then are blended via a ConvNet. Preliminary view warping allows reducing the size of the convolutional kernels and thus the learnable parameters count. We propose a residual encoder-decoder for image blending with a Siamese encoder to further keep the parameters count low. We also contribute a hole inpainting algorithm to fill the disocclusions in the warped views. Our view synthesis experiments on real multiview sequences show better objective image quality than state-of-the-art methods due to fewer artifacts in the synthesized images.
Fichier principal
Vignette du fichier
Hobloss et al. - 2021 - Hybrid dual stream blender for wide baseline view .pdf (16.9 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03330264 , version 1 (06-09-2021)

Identifiants

Citer

Nour Hobloss, Lu Zhang, Stephane Lathuiliere, Marco Cagnazzo, Attilio Fiandrotti. Hybrid dual stream blender for wide baseline view synthesis. Signal Processing: Image Communication, 2021, 97, pp.116366. ⟨10.1016/j.image.2021.116366⟩. ⟨hal-03330264⟩
131 Consultations
21 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More