Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency

Tutkimustuotos › › vertaisarvioitu

Standard

Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency. / Gao, Yuan; Bregovic, Robert ; Gotchev, Atanas.

julkaisussa: IEEE Signal Processing Letters, Vuosikerta 27, 2020, s. 1425-1429.

Tutkimustuotos › › vertaisarvioitu

Bibtex - Lataa

@article{e72f917c7cf94c07b28922fcabab51ac,

title = "Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency",

abstract = "Shearlet Transform (ST) has been instrumental for the Densely-Sampled Light Field (DSLF) reconstruction, as it sparsifies the underlying Epipolar-Plane Images (EPIs). The sought sparsification is implemented through an iterative regularization, which tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which employs ST and cycle consistency. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI-reconstruction and cycle-consistency losses. CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges (⩽ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16-32 pixels) demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ∼ 9x speedup over ST, at least.",

keywords = "cycle consistency, Image-based rendering, light field reconstruction, self-supervision, shearlet transform",

author = "Yuan Gao and Robert Bregovic and Atanas Gotchev",

year = "2020",

doi = "10.1109/LSP.2020.3008082",

language = "English",

volume = "27",

pages = "1425--1429",

journal = "IEEE Signal Processing Letters",

issn = "1070-9908",

publisher = "Institute of Electrical and Electronics Engineers",

}

RIS (suitable for import to EndNote) - Lataa

TY - JOUR

T1 - Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency

AU - Gao, Yuan

AU - Bregovic, Robert

AU - Gotchev, Atanas

PY - 2020

Y1 - 2020

N2 - Shearlet Transform (ST) has been instrumental for the Densely-Sampled Light Field (DSLF) reconstruction, as it sparsifies the underlying Epipolar-Plane Images (EPIs). The sought sparsification is implemented through an iterative regularization, which tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which employs ST and cycle consistency. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI-reconstruction and cycle-consistency losses. CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges (⩽ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16-32 pixels) demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ∼ 9x speedup over ST, at least.

AB - Shearlet Transform (ST) has been instrumental for the Densely-Sampled Light Field (DSLF) reconstruction, as it sparsifies the underlying Epipolar-Plane Images (EPIs). The sought sparsification is implemented through an iterative regularization, which tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which employs ST and cycle consistency. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI-reconstruction and cycle-consistency losses. CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges (⩽ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16-32 pixels) demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ∼ 9x speedup over ST, at least.

KW - cycle consistency

KW - Image-based rendering

KW - light field reconstruction

KW - self-supervision

KW - shearlet transform

U2 - 10.1109/LSP.2020.3008082

DO - 10.1109/LSP.2020.3008082

M3 - Article

VL - 27

SP - 1425

EP - 1429

JO - IEEE Signal Processing Letters

JF - IEEE Signal Processing Letters

SN - 1070-9908

ER -

TUTCRIS