Pázmány Péter Catholic University

Image processing Computer vision

Static point clouds processing

Ref: Benedek Csaba

Background and objectives
Achievements and activities
Deliverables and results

The goal of this part was to use C++ and Point Cloud Library (PCL) on bus stations scenes generated by a LiDAR, to segment pedestrians. This first part was to be familiar with PCL. More especially, learn about the library limits and possibilities before proceed to a real-time application.

I made everything from scratch, to begin with the installation of the environment to the pipeline to segment pedestrians. In this example I used PCL functions to "clean" the point clouds in order to leave only pedestrians. Then I detect each different pedestrian using a cloud density feature.

I haven't made proper metrics as the goal was to learn how to use PCL. Globally, the pipeline worked well (visually) with some difficulties when people were sat on shelters or when they were too close from each other.


Real-time point clouds processing

Background and objectives
Achievements and activities
Deliverables and results

With the prominent arrival of autonomous cars or, more generally, self-driving vehicles, a lot of researches have been made around real-time 3D environement understanding. This would give vehicles the necessary informations they need to decide an action. The challenge is that it has to be accurate and fast enough to be able to react in time in case of crisis.

The application was made in C++ with PCL. I was in charge of finding a solution for pedestrians and cars detection and recognition from a LiDAR's continuous stream. To do so, I reproduced a state-of-the-art article pipeline about 3D real-time detection and add a little geometric based detection to enhance the classification. The article isn't free of charge so I can't share the code or the pipeline. The detection is made by "traditional" image processing while recognition is made by a custom convolutional neural networks (CNN) with few layers to be ran in real-time. I created and trained this custom CNN with Keras to match the article but isn't the proper CNN used in the article.
To resume, in this project I implemented 3D object detection, 3D semantic segmentation, and 3D instance segmentation.

Finally, my application couldn't ran in real-time as my calculation power were lower than the one in the article. Even thought, it didn't run "fluently", it had an accuracy around 70% with the combinaison of the CNN and the geometric based solution. Here are some images extracted from the pipeline with the semantic segmentation, instance detection, and finally, the final result of my application.