Tampere University of Technology

TUTCRIS Research Portal

Joint Sparse Recovery of Misaligned Multimodal Images via Adaptive Local and Nonlocal Cross-Modal Regularization

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Original languageEnglish
Title of host publication2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2019 - Proceedings
Number of pages5
ISBN (Electronic)9781728155494
Publication statusPublished - 1 Dec 2019
Publication typeA4 Article in a conference publication
EventIEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing - Le Gosier, Guadeloupe
Duration: 15 Dec 201918 Dec 2019
Conference number: 8th


ConferenceIEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing
Abbreviated titleCAMSAP
CityLe Gosier


Given few noisy linear measurements of distinct misaligned modalities, we aim at recovering the underlying multimodal image using a sparsity promoting algorithm. Unlike previous multimodal sparse recovery approaches employing side information under the naive assumption of perfect calibration of modalities or of known deformation parameters, we adaptively estimate the deformation parameters from the images separately recovered from the incomplete measurements. We develop a multiscale dense registration method that proceeds alternately by finding block-wise intensity mapping models and a shift vector field which is used to obtain and refine the deformation parameters through a weighted least-squares approximation. The co-registered images are then jointly recovered in a plug-and-play framework where a collaborative filter leverages the local and nonlocal cross-modal correlations inherent to the multimodal image. Our experiments with this fully automatic registration and joint recovery pipeline show a better detection and sharper recovery of fine details which could not be separately recovered.

Downloads statistics

No data available