Tampere University of Technology

TUTCRIS Research Portal

Urban 3D segmentation and modelling from street view images and LiDAR point clouds

Research output: Contribution to journalArticleScientificpeer-review


Original languageEnglish
Pages (from-to)679–694
Number of pages16
JournalMachine Vision and Applications
Issue number7
Publication statusPublished - 2017
Publication typeA1 Journal article-refereed


3D urban maps with semantic labels and metric information are not only essential for the next generation robots such autonomous vehicles and city drones, but also help to visualize and augment local environment in mobile user applications. The machine vision challenge is to generate accurate urban maps from existing data with minimal manual annotation. In this work, we propose a novel methodology that takes GPS registered LiDAR (Light Detection And Ranging) point clouds and street view images as inputs and creates semantic labels for the 3D points clouds using a hybrid of rule-based parsing and learning-based labelling that combine point cloud and photometric features. The rule-based parsing boosts segmentation of simple and large structures such as street surfaces and building facades that span almost 75% of the point cloud data. For more complex structures, such as cars, trees and pedestrians, we adopt boosted decision trees that exploit both structure (LiDAR) and photometric (street view) features. We provide qualitative examples of our methodology in 3D visualization where we construct parametric graphical models from labelled data and in 2D image segmentation where 3D labels are back projected to the street view images. In quantitative evaluation we report classification accuracy and computing times and compare results to competing methods with three popular databases: NAVTEQ True, Paris-Rue-Madame and TLS (terrestrial laser scanned) Velodyne.


  • LiDAR, Point cloud, Robotics, Semantic segmentation, Street view, Urban 3D

Publication forum classification

Field of science, Statistics Finland

Downloads statistics

No data available