Tampere University of Technology

TUTCRIS Research Portal

Portrait instance segmentation for mobile devices

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Original languageEnglish
Title of host publication2019 IEEE International Conference on Multimedia and Expo, ICME 2019
Number of pages6
ISBN (Electronic)9781538695524
Publication statusPublished - 1 Jul 2019
Publication typeA4 Article in a conference publication
EventIEEE International Conference on Multimedia and Expo - Shanghai, China
Duration: 8 Jul 201912 Jul 2019


ConferenceIEEE International Conference on Multimedia and Expo


Accurate and efficient portrait instance segmentation has become a crucial enabler for many multimedia applications on mobile devices. We present a novel convolutional neural network (CNN) architecture to explicitly address the long standing problems in portrait segmentation, i.e., semantic coherence and boundary localization. Specifically, we propose a cross-granularity categorical attention mechanism leveraging the deep supervisions to close the semantic gap of CNN feature hierarchy by imposing consistent category-oriented information across layers. Furthermore, a cross-granularity boundary enhancement module is proposed to boost the boundary awareness of deep layers by integrating the shape context cues from shallow layers of the network. We further propose a novel and efficient non-parametric affinity model to achieve efficient instance segmentation on mobile devices. We present a portrait image dataset with instance level annotations dedicated to evaluating portrait instance segmentation algorithms. We evaluate our approach on challenging datasets which obtains state-of-the-art results.


  • Convolutional neural networks, Instance segmentation, Portrait segmentation, Semantic segmentation

Publication forum classification

Field of science, Statistics Finland