Tampere University of Technology

TUTCRIS Research Portal

Unsupervised Facial Image De-occlusion with Optimized Deep Generative Models

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Original languageEnglish
Title of host publication2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA)
ISBN (Electronic)978-1-5386-6428-5
ISBN (Print)978-1-5386-6429-2
Publication statusPublished - Nov 2018
Publication typeA4 Article in a conference publication
EventInternational Conference on Image Processing Theory, Tools and Applications -
Duration: 1 Jan 1900 → …


ConferenceInternational Conference on Image Processing Theory, Tools and Applications
Period1/01/00 → …


In recent years, Generative Adversarial Networks (GANs) or various types of Auto-Encoders (AEs) have gained attention on facial image de-occlusion and/or in-painting tasks. In this paper, we propose a novel unsupervised technique to remove occlusion from facial images and complete the occluded parts simultaneously with optimized Deep Convolutional Generative Adversarial Networks (DCGANs) in an iterative way. Generally, GANs, as generative models, can estimate the distribution of images using a generator and a discriminator. DCGANs, as its variant, are proposed to conquer its instability during training. Existing facial image in-painting methods manually define a block of pixels as the missing part and the potential content of this block is semantically generated using generative models, such as GANs or AEs. In our method, a mask is inferred from an occluded facial image using a novel loss function, and then this mask is utilized to in-paint the occlusions automatically by pre-trained DCGANs. We evaluate the performance of our method on facial images with various occlusions, such as sunglasses and scarves. The experiments demonstrate that our method can effectively detect certain kinds of occlusions and complete the occluded parts in an unsupervised manner.

Publication forum classification

Field of science, Statistics Finland