SlideShare uma empresa Scribd logo

Sensor fusion of LiDAR and Camera for real time object detection - talk version

•
•
NECST Lab @ Politecnico di Milano
NECST Lab @ Politecnico di Milano

Autonomous vehicles primary need is to have a robust and precise perception of the surrounding environment. To achieve accurate results in autonomous driving a combination of several different sensors is used. This technique is commonly known as sensor fusion. However, a critical aspect of autonomous driving perception is also the reactivity: we do not only need to perform sensor fusion properly, but we also have to respect real time requirements. The latter are fundamental to guarantee predictability of the system, avoid anomalous situations and prevent hazards. Performing fusion online is not trivial because sensors returns a lot of data and process them may be time consuming. The solution we present is an alignment of the sensors, which combines information from LiDAR and cameras, that is performed once in preprocessing and allow us to exploit the precomputed matching in real time.

Indústria automotiva
1 de 22
Baixar para ler offline
Sensor fusion for autonomous driving perception Bagni Fabio 228594@studenti.unimore .it San Francisco - 15 May 2019 1
Fabio Bagni San Francisco - 15 May 2019 Autonomous driving perception Precision and reactivity Risks of accidents, traffic laws respect. 2
Fabio Bagni San Francisco - 15 May 2019 Embedded platforms 3
Fabio Bagni San Francisco - 15 May 2019 Goals Real time 3D detection ● High performances ○ required 10 Hz frequency ● High precision ● Real time on embedded platforms 4
Fabio Bagni San Francisco - 15 May 2019 UNIMORE HiPeRT prototype 5
Fabio Bagni San Francisco - 15 May 2019 Sensors ● LiDAR ○ 3D point cloud ● Cameras ○ images Heterogeneous outputs 6
Fabio Bagni San Francisco - 15 May 2019 Sensor fusion What is sensor fusion: ● Matching of different sensors views Why sensor fusion: ● Heterogeneous information of environment 7
Fabio Bagni San Francisco - 15 May 2019 Fields of view LiDAR: 360 degrees around the vehicle Camera: wide angle, disposed do cover 360 degrees. 8 Cameras LiDAR
Fabio Bagni San Francisco - 15 May 2019 Deep learning approach Camera: ● Bird’s Eye View estimation ● Object detection LiDAR ● Bird’s Eye View calculation ● Object detection 9 Image Point cloud 3D detection 3D detection Fusion
Fabio Bagni San Francisco - 15 May 2019 Deep learning approach time Image stitching for multiple camera fusion 10 ➔ Under10 Hz Notenoughresponsive 1. Ming Liang, Bin Yang, Shenlong Wang and Raquel Urtasun: Deep Continuous Fusion for Multi-Sensor3D ObjectDetection.In: ECCV (2018).
Fabio Bagni San Francisco - 15 May 2019 Static alignment sensor fusion Alignment of LiDAR with each camera individually. 11 Calibration Preprocessing
Fabio Bagni San Francisco - 15 May 2019 Features extraction Matching is made by a calibration with a perforated panel. Holes are detected by all sensors. Holes’ centers are used as features to be matched. 12
Fabio Bagni San Francisco - 15 May 2019 Cylindrical projection Fields of view matching is made projecting sensors’ outputs on the surface of a cylinder. 13
Fabio Bagni San Francisco - 15 May 2019 Projections complexity LiDAR cylindrical projection: Depthmap O(n) : one operation per point 14 Camera cylindrical projection O(n) : one operation per pixel PARALLELIZABLE
Fabio Bagni San Francisco - 15 May 2019 Calibration results 15
Fabio Bagni San Francisco - 15 May 2019 Colored points Alignment of each point with the corresponding pixel on the image. 16
Fabio Bagni San Francisco - 15 May 2019 Object detection Neural network for object detection on camera frame Clustering of LiDAR points Bounding boxes matching 17
Fabio Bagni San Francisco - 15 May 2019 Object detection results 18
Fabio Bagni San Francisco - 15 May 2019 Object detection average time 19 LiDAR output : 10 Hz Time limit : 100 ms
Thanks for your attention Fabio Bagni - 228594@studenti.unimore.it 20 Fabio Bagni San Francisco - 15 May 2019
21
Fabio Bagni San Francisco - 15 May 2019 Homography Homography allow to translate camera projection plane on the LiDAR projection plane. This method make cylindrical surfaces match. 22

Recomendados

fusion of Camera and lidar for autonomous driving II
fusion of Camera and lidar for autonomous driving IIfusion of Camera and lidar for autonomous driving II
fusion of Camera and lidar for autonomous driving IIYu Huang
 
fusion of Camera and lidar for autonomous driving I
fusion of Camera and lidar for autonomous driving Ifusion of Camera and lidar for autonomous driving I
fusion of Camera and lidar for autonomous driving IYu Huang
 
Lidar for Autonomous Driving II (via Deep Learning)
Lidar for Autonomous Driving II (via Deep Learning)Lidar for Autonomous Driving II (via Deep Learning)
Lidar for Autonomous Driving II (via Deep Learning)Yu Huang
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detectionBrodmann17
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksUsman Qayyum
 
Driver drowsiness detection
Driver drowsiness detectionDriver drowsiness detection
Driver drowsiness detectionConnecting Point
 
Computer Vision
Computer VisionComputer Vision
Computer VisionAmeer Mohamed Rajah
 
Object tracking presentation
Object tracking presentationObject tracking presentation
Object tracking presentationMrsShwetaBanait1
 

Recomendados

fusion of Camera and lidar for autonomous driving II
fusion of Camera and lidar for autonomous driving IIfusion of Camera and lidar for autonomous driving II
fusion of Camera and lidar for autonomous driving IIYu Huang
 
fusion of Camera and lidar for autonomous driving I
fusion of Camera and lidar for autonomous driving Ifusion of Camera and lidar for autonomous driving I
fusion of Camera and lidar for autonomous driving IYu Huang
 
Lidar for Autonomous Driving II (via Deep Learning)
Lidar for Autonomous Driving II (via Deep Learning)Lidar for Autonomous Driving II (via Deep Learning)
Lidar for Autonomous Driving II (via Deep Learning)Yu Huang
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detectionBrodmann17
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksUsman Qayyum
 
Driver drowsiness detection
Driver drowsiness detectionDriver drowsiness detection
Driver drowsiness detectionConnecting Point
 
Computer Vision
Computer VisionComputer Vision
Computer VisionAmeer Mohamed Rajah
 
Object tracking presentation
Object tracking presentationObject tracking presentation
Object tracking presentationMrsShwetaBanait1
 
Traffic sign detection
Traffic sign detectionTraffic sign detection
Traffic sign detectionAvijit Rai
 
LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)Yu Huang
 
3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -Kazuyuki Miyazawa
 
Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detectionWenjing Chen
 
=SLAM ppt.pdf
=SLAM ppt.pdf=SLAM ppt.pdf
=SLAM ppt.pdfusmanarif88
 
3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving IIYu Huang
 
Introduction of slam
Introduction of slamIntroduction of slam
Introduction of slamHung-Chih Chang
 
Depth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep LearningDepth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep LearningYu Huang
 
Human Action Recognition
Human Action RecognitionHuman Action Recognition
Human Action RecognitionNAVER Engineering
 
Object detection
Object detectionObject detection
Object detectionROUSHAN RAJ KUMAR
 
Introduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable RenderingIntroduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable RenderingPreferred Networks
 
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), FooladSuperpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), FooladShima Foolad
 
Image segmentation
Image segmentationImage segmentation
Image segmentationKuppusamy P
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentationAshwinBicholiya
 
"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from Cadence"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from CadenceEdge AI and Vision Alliance
 
Image captioning
Image captioningImage captioning
Image captioningMuhammad Zbeedat
 
Computer Vision for Beginners
Computer Vision for BeginnersComputer Vision for Beginners
Computer Vision for BeginnersSanghamitra Deb
 
Vehicle detection
Vehicle detectionVehicle detection
Vehicle detectionEvangelos Skodras
 
Object Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkObject Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkNader Karimi
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 
Mesticheria Team - WiiReflex
Mesticheria Team - WiiReflexMesticheria Team - WiiReflex
Mesticheria Team - WiiReflexNECST Lab @ Politecnico di Milano
 
Punto e virgola Team - Stressometro
Punto e virgola Team - StressometroPunto e virgola Team - Stressometro
Punto e virgola Team - StressometroNECST Lab @ Politecnico di Milano
 

Mais conteúdo relacionado

Mais procurados

Traffic sign detection
Traffic sign detectionTraffic sign detection
Traffic sign detectionAvijit Rai
 
LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)Yu Huang
 
3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -Kazuyuki Miyazawa
 
Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detectionWenjing Chen
 
3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving IIYu Huang
 
Depth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep LearningDepth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep LearningYu Huang
 
Introduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable RenderingIntroduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable RenderingPreferred Networks
 
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), FooladSuperpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), FooladShima Foolad
 
Image segmentation
Image segmentationImage segmentation
Image segmentationKuppusamy P
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentationAshwinBicholiya
 
"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from Cadence"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from CadenceEdge AI and Vision Alliance
 
Computer Vision for Beginners
Computer Vision for BeginnersComputer Vision for Beginners
Computer Vision for BeginnersSanghamitra Deb
 
Object Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkObject Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkNader Karimi
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 

Mais procurados (20)

Traffic sign detection
Traffic sign detectionTraffic sign detection
Traffic sign detection
 
LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)LiDAR-based Autonomous Driving III (by Deep Learning)
LiDAR-based Autonomous Driving III (by Deep Learning)
 
3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -3D Perception for Autonomous Driving - Datasets and Algorithms -
3D Perception for Autonomous Driving - Datasets and Algorithms -
 
Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detection
 
=SLAM ppt.pdf
=SLAM ppt.pdf=SLAM ppt.pdf
=SLAM ppt.pdf
 
3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II3-d interpretation from single 2-d image for autonomous driving II
3-d interpretation from single 2-d image for autonomous driving II
 
Introduction of slam
Introduction of slamIntroduction of slam
Introduction of slam
 
Depth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep LearningDepth Fusion from RGB and Depth Sensors by Deep Learning
Depth Fusion from RGB and Depth Sensors by Deep Learning
 
Human Action Recognition
Human Action RecognitionHuman Action Recognition
Human Action Recognition
 
Object detection
Object detectionObject detection
Object detection
 
Introduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable RenderingIntroduction to 3D Computer Vision and Differentiable Rendering
Introduction to 3D Computer Vision and Differentiable Rendering
 
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), FooladSuperpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
Superpixel algorithms (whatershed, mean-shift, SLIC, BSLIC), Foolad
 
Image segmentation
Image segmentationImage segmentation
Image segmentation
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentation
 
"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from Cadence"Fundamentals of Monocular SLAM," a Presentation from Cadence
"Fundamentals of Monocular SLAM," a Presentation from Cadence
 
Image captioning
Image captioningImage captioning
Image captioning
 
Computer Vision for Beginners
Computer Vision for BeginnersComputer Vision for Beginners
Computer Vision for Beginners
 
Vehicle detection
Vehicle detectionVehicle detection
Vehicle detection
 
Object Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkObject Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning Framework
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
 

Mais de NECST Lab @ Politecnico di Milano

Embedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposingEmbedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposingNECST Lab @ Politecnico di Milano
 
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...NECST Lab @ Politecnico di Milano
 
EMPhASIS - An EMbedded Public Attention Stress Identification System
EMPhASIS - An EMbedded Public Attention Stress Identification System EMPhASIS - An EMbedded Public Attention Stress Identification System
EMPhASIS - An EMbedded Public Attention Stress Identification SystemNECST Lab @ Politecnico di Milano
 
Maeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matchingMaeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matchingNECST Lab @ Politecnico di Milano
 

Mais de NECST Lab @ Politecnico di Milano (20)

Mesticheria Team - WiiReflex
Mesticheria Team - WiiReflexMesticheria Team - WiiReflex
Mesticheria Team - WiiReflex
 
Punto e virgola Team - Stressometro
Punto e virgola Team - StressometroPunto e virgola Team - Stressometro
Punto e virgola Team - Stressometro
 
BitIt Team - Stay.straight
BitIt Team - Stay.straight BitIt Team - Stay.straight
BitIt Team - Stay.straight
 
BabYodini Team - Talking Gloves
BabYodini Team - Talking GlovesBabYodini Team - Talking Gloves
BabYodini Team - Talking Gloves
 
printf("Nome Squadra"); Team - NeoTon
printf("Nome Squadra"); Team - NeoTonprintf("Nome Squadra"); Team - NeoTon
printf("Nome Squadra"); Team - NeoTon
 
BlackBoard Team - Motion Tracking Platform
BlackBoard Team - Motion Tracking PlatformBlackBoard Team - Motion Tracking Platform
BlackBoard Team - Motion Tracking Platform
 
#include<brain.h> Team - HomeBeatHome
#include<brain.h> Team - HomeBeatHome#include<brain.h> Team - HomeBeatHome
#include<brain.h> Team - HomeBeatHome
 
Flipflops Team - Wave U
Flipflops Team - Wave UFlipflops Team - Wave U
Flipflops Team - Wave U
 
Bug(atta) Team - Little Brother
Bug(atta) Team - Little BrotherBug(atta) Team - Little Brother
Bug(atta) Team - Little Brother
 
#NECSTCamp: come partecipare
#NECSTCamp: come partecipare#NECSTCamp: come partecipare
#NECSTCamp: come partecipare
 
NECSTCamp101@2020.10.1
NECSTCamp101@2020.10.1NECSTCamp101@2020.10.1
NECSTCamp101@2020.10.1
 
NECSTLab101 2020.2021
NECSTLab101 2020.2021NECSTLab101 2020.2021
NECSTLab101 2020.2021
 
TreeHouse, nourish your community
TreeHouse, nourish your communityTreeHouse, nourish your community
TreeHouse, nourish your community
 
TiReX: Tiled Regular eXpressionsmatching architecture
TiReX: Tiled Regular eXpressionsmatching architectureTiReX: Tiled Regular eXpressionsmatching architecture
TiReX: Tiled Regular eXpressionsmatching architecture
 
Embedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposingEmbedding based knowledge graph link prediction for drug repurposing
Embedding based knowledge graph link prediction for drug repurposing
 
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
PLASTER - PYNQ-based abandoned object detection using a map-reduce approach o...
 
EMPhASIS - An EMbedded Public Attention Stress Identification System
EMPhASIS - An EMbedded Public Attention Stress Identification System EMPhASIS - An EMbedded Public Attention Stress Identification System
EMPhASIS - An EMbedded Public Attention Stress Identification System
 
Luns - Automatic lungs segmentation through neural network
Luns - Automatic lungs segmentation through neural networkLuns - Automatic lungs segmentation through neural network
Luns - Automatic lungs segmentation through neural network
 
BlastFunction: How to combine Serverless and FPGAs
BlastFunction: How to combine Serverless and FPGAsBlastFunction: How to combine Serverless and FPGAs
BlastFunction: How to combine Serverless and FPGAs
 
Maeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matchingMaeve - Fast genome analysis leveraging exact string matching
Maeve - Fast genome analysis leveraging exact string matching
 

Último

John deere 425 445 455 Maitenance Manual
John deere 425 445 455 Maitenance ManualJohn deere 425 445 455 Maitenance Manual
John deere 425 445 455 Maitenance ManualExcavator
 
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...amitlee9823
 
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...Delhi Call girls
 
John Deere 335 375 385 435 Service Repair Manual
John Deere 335 375 385 435 Service Repair ManualJohn Deere 335 375 385 435 Service Repair Manual
John Deere 335 375 385 435 Service Repair ManualExcavator
 
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一opyff
 
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一ozave
 
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理ezgenuh
 
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!AutoScandia
 
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...anilsa9823
 
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...amitlee9823
 
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...amitlee9823
 
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国ezgenuh
 
Greenery-Palette Pitch Deck by Slidesgo.pptx
Greenery-Palette Pitch Deck by Slidesgo.pptxGreenery-Palette Pitch Deck by Slidesgo.pptx
Greenery-Palette Pitch Deck by Slidesgo.pptxzohiiimughal286
 
What Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop WorkingWhat Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop WorkingBruce Cox Imports
 
9990611130 Find & Book Russian Call Girls In Vijay Nagar
9990611130 Find & Book Russian Call Girls In Vijay Nagar9990611130 Find & Book Russian Call Girls In Vijay Nagar
9990611130 Find & Book Russian Call Girls In Vijay NagarGenuineGirls
 
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagardollysharma2066
 
design a four cylinder internal combustion engine
design a four cylinder internal combustion enginedesign a four cylinder internal combustion engine
design a four cylinder internal combustion enginepiyushsingh943161
 
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Callshivangimorya083
 

Último (20)

John deere 425 445 455 Maitenance Manual
John deere 425 445 455 Maitenance ManualJohn deere 425 445 455 Maitenance Manual
John deere 425 445 455 Maitenance Manual
 
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Kanakapura Road Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
 
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
Call Girls in Malviya Nagar Delhi 💯 Call Us 🔝9205541914 🔝( Delhi) Escorts Ser...
 
John Deere 335 375 385 435 Service Repair Manual
John Deere 335 375 385 435 Service Repair ManualJohn Deere 335 375 385 435 Service Repair Manual
John Deere 335 375 385 435 Service Repair Manual
 
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一
如何办理女王大学毕业证（QU毕业证书）成绩单原版一比一
 
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一
如何办理麦考瑞大学毕业证（MQU毕业证书）成绩单原版一比一
 
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
一比一原版(PU学位证书)普渡大学毕业证学历认证加急办理
 
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!
Why Won't Your Subaru Key Come Out Of The Ignition Find Out Here!
 
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...
Lucknow 💋 (Genuine) Escort Service Lucknow | Service-oriented sexy call girls...
 
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Bangalore Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
 
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
Sanjay Nagar Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalor...
 
(INDIRA) Call Girl Nashik Call Now 8617697112 Nashik Escorts 24x7
(INDIRA) Call Girl Nashik Call Now 8617697112 Nashik Escorts 24x7(INDIRA) Call Girl Nashik Call Now 8617697112 Nashik Escorts 24x7
(INDIRA) Call Girl Nashik Call Now 8617697112 Nashik Escorts 24x7
 
Call Girls in Shri Niwas Puri Delhi 💯Call Us 🔝9953056974🔝
Call Girls in Shri Niwas Puri Delhi 💯Call Us 🔝9953056974🔝Call Girls in Shri Niwas Puri Delhi 💯Call Us 🔝9953056974🔝
Call Girls in Shri Niwas Puri Delhi 💯Call Us 🔝9953056974🔝
 
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国
一比一原版(UVic学位证书)维多利亚大学毕业证学历认证买留学回国
 
Greenery-Palette Pitch Deck by Slidesgo.pptx
Greenery-Palette Pitch Deck by Slidesgo.pptxGreenery-Palette Pitch Deck by Slidesgo.pptx
Greenery-Palette Pitch Deck by Slidesgo.pptx
 
What Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop WorkingWhat Could Cause Your Subaru's Touch Screen To Stop Working
What Could Cause Your Subaru's Touch Screen To Stop Working
 
9990611130 Find & Book Russian Call Girls In Vijay Nagar
9990611130 Find & Book Russian Call Girls In Vijay Nagar9990611130 Find & Book Russian Call Girls In Vijay Nagar
9990611130 Find & Book Russian Call Girls In Vijay Nagar
 
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
83778-77756 ( HER.SELF ) Brings Call Girls In Laxmi Nagar
 
design a four cylinder internal combustion engine
design a four cylinder internal combustion enginedesign a four cylinder internal combustion engine
design a four cylinder internal combustion engine
 
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Saket 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
 

Sensor fusion of LiDAR and Camera for real time object detection - talk version

  • 1. Sensor fusion for autonomous driving perception Bagni Fabio 228594@studenti.unimore .it San Francisco - 15 May 2019 1
  • 2. Fabio Bagni San Francisco - 15 May 2019 Autonomous driving perception Precision and reactivity Risks of accidents, traffic laws respect. 2
  • 3. Fabio Bagni San Francisco - 15 May 2019 Embedded platforms 3
  • 4. Fabio Bagni San Francisco - 15 May 2019 Goals Real time 3D detection ● High performances ○ required 10 Hz frequency ● High precision ● Real time on embedded platforms 4
  • 5. Fabio Bagni San Francisco - 15 May 2019 UNIMORE HiPeRT prototype 5
  • 6. Fabio Bagni San Francisco - 15 May 2019 Sensors ● LiDAR ○ 3D point cloud ● Cameras ○ images Heterogeneous outputs 6
  • 7. Fabio Bagni San Francisco - 15 May 2019 Sensor fusion What is sensor fusion: ● Matching of different sensors views Why sensor fusion: ● Heterogeneous information of environment 7
  • 8. Fabio Bagni San Francisco - 15 May 2019 Fields of view LiDAR: 360 degrees around the vehicle Camera: wide angle, disposed do cover 360 degrees. 8 Cameras LiDAR
  • 9. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach Camera: ● Bird’s Eye View estimation ● Object detection LiDAR ● Bird’s Eye View calculation ● Object detection 9 Image Point cloud 3D detection 3D detection Fusion
  • 10. Fabio Bagni San Francisco - 15 May 2019 Deep learning approach time Image stitching for multiple camera fusion 10 ➔ Under10 Hz Notenoughresponsive 1. Ming Liang, Bin Yang, Shenlong Wang and Raquel Urtasun: Deep Continuous Fusion for Multi-Sensor3D ObjectDetection.In: ECCV (2018).
  • 11. Fabio Bagni San Francisco - 15 May 2019 Static alignment sensor fusion Alignment of LiDAR with each camera individually. 11 Calibration Preprocessing
  • 12. Fabio Bagni San Francisco - 15 May 2019 Features extraction Matching is made by a calibration with a perforated panel. Holes are detected by all sensors. Holes’ centers are used as features to be matched. 12
  • 13. Fabio Bagni San Francisco - 15 May 2019 Cylindrical projection Fields of view matching is made projecting sensors’ outputs on the surface of a cylinder. 13
  • 14. Fabio Bagni San Francisco - 15 May 2019 Projections complexity LiDAR cylindrical projection: Depthmap O(n) : one operation per point 14 Camera cylindrical projection O(n) : one operation per pixel PARALLELIZABLE
  • 15. Fabio Bagni San Francisco - 15 May 2019 Calibration results 15
  • 16. Fabio Bagni San Francisco - 15 May 2019 Colored points Alignment of each point with the corresponding pixel on the image. 16
  • 17. Fabio Bagni San Francisco - 15 May 2019 Object detection Neural network for object detection on camera frame Clustering of LiDAR points Bounding boxes matching 17
  • 18. Fabio Bagni San Francisco - 15 May 2019 Object detection results 18
  • 19. Fabio Bagni San Francisco - 15 May 2019 Object detection average time 19 LiDAR output : 10 Hz Time limit : 100 ms
  • 20. Thanks for your attention Fabio Bagni - 228594@studenti.unimore.it 20 Fabio Bagni San Francisco - 15 May 2019
  • 21. 21
  • 22. Fabio Bagni San Francisco - 15 May 2019 Homography Homography allow to translate camera projection plane on the LiDAR projection plane. This method make cylindrical surfaces match. 22