TUTCRIS - Tampereen teknillinen yliopisto

TUTCRIS

End-to-end learning for video frame compression with self-attention

Tutkimustuotosvertaisarvioitu

Yksityiskohdat

AlkuperäiskieliEnglanti
OtsikkoProceedings - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020
KustantajaIEEE
Sivut580-584
Sivumäärä5
ISBN (elektroninen)9781728193601
ISBN (painettu)978-1-7281-9361-8
DOI - pysyväislinkit
TilaJulkaistu - 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops - Virtual, Online, Yhdysvallat
Kesto: 14 kesäkuuta 202019 kesäkuuta 2020

Julkaisusarja

NimiIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (painettu)2160-7508
ISSN (elektroninen)2160-7516

Conference

ConferenceIEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops
MaaYhdysvallat
KaupunkiVirtual, Online
Ajanjakso14/06/2019/06/20

Tiivistelmä

One of the core components of conventional (i.e., non-learned) video codecs consists of predicting a frame from a previously-decoded frame, by leveraging temporal correlations. In this paper, we propose an end-to-end learned system for compressing video frames. Instead of relying on pixel-space motion (as with optical flow), our system learns deep embeddings of frames and encodes their difference in latent space. At decoder-side, an attention mechanism is designed to attend to the latent space of frames to decide how different parts of the previous and current frame are combined to form the final predicted current frame. Spatially-varying channel allocation is achieved by using importance masks acting on the feature-channels. The model is trained to reduce the bitrate by minimizing a loss on importance maps and a loss on the probability output by a context model for arithmetic coding. In our experiments, we show that the proposed system achieves high compression rates and high objective visual quality as measured by MS-SSIM and PSNR. Furthermore, we provide ablation studies where we highlight the contribution of different components.