Joint Human Detection from On-Board and Off-Board Cameras

•

0 gostou•986 visualizações

Justas Miseikis

Tecnologia Negócios

Joint Human Detection from On-Board
and Off-Board Cameras
Justinas Mišeikis and Paulo V. K. Borges

Problem
Various vehicles and people share the same environment.

Unfortunately, in these conditions accidents occur.

Problem

Can we use technology to
prevent them?

Related Work
• Pedestrian detection using vehicle mounted
stereo cameras
• Crowd tracking in enclosed environments
• People tracking using ﬁxed cameras and
driver warning if a person is on the path
• Same object identiﬁcation using off-board
cameras and not localized mobile camera

Our Method Overview
Off-Board camera In world coordinates

Pos relative to On-Board camera
the vehicle detection

Off-Board Cameras
• Fixed cameras
• MOG2 background segmentation
• Noise ﬁltering
• Blob detection
• Size and dimensions ﬁltering
• Feet position - bottom centre
point of the blob

People Detector - HOG
• Popular method for people detection
• Works well in cluttered environments
• Based on distribution of intensity gradients
or edge directions.
• Descriptor created from many samples

On-Board Cameras
Full Image Analysis

The camera view is split according A: Whole Image analysis - person not
to the distance from the camera: detected
blue (3-7 meters), green (7-12 B: Area split method - person detected
meters), yellow (12-20 meters).

Targeted Analysis

Expected feet Detected
position from person
ground plane
homography

HOG search area
Size calculated using the expected person height

Data Fusion
Off-Board Cam A Off-Board Cam B

Detected On-Board Cam - HOG
people
positions
Position Fusion

Final Pedestrian Position

Data Fusion
Position sensor fusion

6

HOG detector variance

Estimated distance error (meters)
Measurement Error
5
Polynomial Error Estimation

estimation depending 4

on the object’s distance
3

2

from the camera 1

0
2 3 4 5 6 7 8 9 10 11 12
Distance from the camera (meters)

Extended Kalman Filter for sensor fusion and tracking

Implementation
• C++
• Standard Intel laptop + desktop running in
parallel
• cvBlob library

Experiments
30m by 30m industrial site
Test vehicle - HMC
2 Off-Board, 1 On-Board cam
Three 4-6 minute runs
1-2 People walking around
Vehicle static and moving

Results
FP % FN %
Localized Analysis 4.41 3.66
Full Image Analysis 95.53 2.40

• Works real-time
• Background Segmentation - 10-15 FPS
• HOG detectors - 5 FPS

Progress
• Additional front facing cameras, 4 in total
- Two side cameras
- Wide angle camera for close proximity
- Far proximity front camera
• People tracking
• HOG performed on GPU GeForce GT640

Progress
Close Prox Cam

Danger Zone -
reduced to around
20 cm from the
front of the vehicle
Far Prox Cam

Mais conteúdo relacionado

Semelhante a Joint Human Detection from On-Board and Off-Board Cameras

(Reading Group) First Results in Detecting and Avoiding Frontal Obstacles fro...

Mohamed Elawady

Online Camer Calibration

Fei-Fei Zheng

Landmines Detection by Robots presentation

Ahmed Abdelaziz

F011113741

IOSR Journals

IOSR Journal of Electronics and Communication Engineering(IOSR-JECE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of electronics and communication engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in electronics and communication engineering. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.

Real Time Detection of Moving Object Based on Fpga

iosrjce

It is a wearable device. It has a camera, and it detects all living and non living object. This module detects moving object also. It is made with raspberry pi 3, and a camera. One headphone connect with raspberry pi. When this module detects items, it gave a sound output through headphone. Hence the blind man know that item, which is in-front of him or her. We made it in very low budget, and it is very helpful for visually challenged people. And the Blind stick help him to detect obstacles.

Development of wearable object detection system & blind stick for visuall...

Arkadev Kundu

optimizing-blind-navigation-machine-learning-for-object-detection-and-distanc...

ranavh003

Trail following and obstacle avoidance

민재 정

We present in this paper a very fine method for determining the location of a ground based target when viewed from an Unmanned Aerial Vehicle (UAV). By determining the pixel coordinates on the video frame and by using a range finder the target’s geo-location is determined in the North- East-Down (NED) frame. The contribution of this method is that the target can be localized to within 9m when view from an altitude of 2500m and down to 1m from an altitude of 100m. This method offers a highly versatile tracking and geolocalisation technique that has very good number of advantages over the previously suggested methods. Some of the key factors that differentiate our method from its predecessors are: 1) Day and night time operation 2) All weather operation 3) Highly accurate positioning of target in terms of latitude-longitude (GPS) and altitude. 4) Automatic gimbaled operation of the camera once target is locked 5) Tracking is possible even when the target stops moving 6) Independent of target (moving or stationary) 7) No terrain database is required 8) Instantaneous target geolocalisation is possible

Image Fusion of Video Images and Geo-localization for UAV Applications

IDES Editor

Perspective Multiscale Detection and Tracking of Persons

Marcos Nieto

Emerging vision technologies

Qualcomm Research

Join us to hear how we created Kuri, the world’s most adorable home robot, and how cuteness and machine learning algorithms come together in creating a robot that people are excited to bring into their homes. Kuri uses embedded, deep-learning-based algorithms for face, pet, and person detection, as well as for place recognition (for mapping and localization). Such algorithms are crucial for enabling her endearing interactions with people, as well as her ability to be the family videographer and entertainer. Cute behaviors also enable Kuri to subtly and smoothly gather necessary data for training and inference, providing a significant benefit in improving core functionality for adorable home robots.

Kaijen Hsiao at AI Frontiers: Adorable Intelligence

AI Frontiers

For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-osterwood-tue For more information about embedded vision, please visit: http://www.embedded-vision.com Chris Osterwood, Founder and CEO of Capable Robot Components, presents the "How to Choose a 3D Vision Sensor" tutorial at the May 2019 Embedded Vision Summit. Designers of autonomous vehicles, robots and many other systems are faced with a critical challenge: Which 3D vision sensor technology to use? There are a wide variety of sensors on the market, employing modalities including passive stereo, active stereo, time of flight, 2D and 3D lasers and monocular approaches. This talk provides an overview of 3D vision sensor technologies and their capabilities and limitations, based on Osterwood's experience selecting the right 3D technology and sensor for a diverse range of autonomous robot designs. There is no perfect sensor technology and no perfect sensor, but there is always a sensor which best aligns with the requirements of your application—you just need to find it. Osterwood describes a quantitative and qualitative evaluation process for 3D vision sensors, including testing processes using both controlled environments and field testing, and some surprising characteristics and limitations he's uncovered through that testing.

"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components

Edge AI and Vision Alliance

IRJET - Automatic Gun Control using Motion Detection System

IRJET Journal

Microparticle characterization using digital holography

Pranit Malgaonkar

Microparticle characterization using digital holography

Pranit Malgaonkar

The extraction and analysis of human gait characteristics using image sequences are currently an intense area of research. Identifying individuals using biometric methods has recently gained growing interest from computer vision researchers for security purposes at places like airport, banks etc. Gait recognition aims essentially to address this problem by identifying people at a distance based on the way they walk i.e., by tracking a number of feature points or gait signatures. We describe a new model-based feature extraction analysis is presented using Hough transform technique that helps to read the essential parameters used to generate gait signatures that automatically extracts and describes human gait for recognition. In the preprocessing steps, the picture frames taken from video sequences are given as input to Canny edge detection algorithm which helps to detect edges of the image by extracting foreground from background also it reduces the noise using Gaussian filter. The output from edge detection is given as input to the Hough transform. Using the Hough transform image, a clear line based model is designed to extract gait signatures. A major difficulty of the existing gait signature extraction methods are the good tracking the requisite feature points. In the proposed work, we have used five parameters to successfully extract the gait signatures. It is observed that when the camera is placed at 90 and 270 degrees, all the parameters used in the proposed work are clearly visible. The efficiency of the model is tested on a variety of body position and stride parameters recovered in different viewing conditions on a database consisting of 20 subjects walking at both an angled and frontal-parallel view with respect to the camera, both indoors and outdoors and find the method to be highly successful. The test results show good clarity rates, with a high level of confidence and it is suggested that the algorithm reported here could form the basis of a robust system for monitoring of gait.

A Parametric Approach to Gait Signature Extraction for Human Motion Identific...

CSCJournals

NoxEye.pptx

MathanE5

Applications of Artificial Intelligence

Appstud

Q180305116119

IOSR Journals

Semelhante a Joint Human Detection from On-Board and Off-Board Cameras (20)

(Reading Group) First Results in Detecting and Avoiding Frontal Obstacles fro...

Online Camer Calibration

Landmines Detection by Robots presentation

F011113741

Real Time Detection of Moving Object Based on Fpga

Development of wearable object detection system & blind stick for visuall...

optimizing-blind-navigation-machine-learning-for-object-detection-and-distanc...

Trail following and obstacle avoidance

Image Fusion of Video Images and Geo-localization for UAV Applications

Perspective Multiscale Detection and Tracking of Persons

Emerging vision technologies

Kaijen Hsiao at AI Frontiers: Adorable Intelligence

"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components

IRJET - Automatic Gun Control using Motion Detection System

Microparticle characterization using digital holography

A Parametric Approach to Gait Signature Extraction for Human Motion Identific...

NoxEye.pptx

Applications of Artificial Intelligence

Q180305116119

Mais de Justas Miseikis

Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and ...

Justas Miseikis

Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D P...

Justas Miseikis

With 3D sensing becoming cheaper, environment- aware and visually-guided robot arms capable of safely working in collaboration with humans will become common. However, a reliable calibration is needed, both for camera internal calibration, as well as Eye-to-Hand calibration, to make sure the whole system functions correctly. We present a framework, using a novel combination of well proven methods, allowing a quick automatic calibration for the integration of systems consisting of the robot and a varying number of 3D cameras by using a standard checkerboard calibration grid. Our approach allows a quick camera-to-robot recalibration after any changes to the setup, for example when cameras or robot have been repositioned. Modular design of the system ensures flexibility regarding a number of sensors used as well as different hardware choices. The framework has been proven to work by practical experiments to analyze the quality of the calibration versus the number of positions of the checkerboard used for each of the calibration procedures.

Automatic Calibration of a Robot Manipulator and Multi 3D Camera System

Justas Miseikis

Electric vehicles (EVs) and plug-in hybrid vehicles (PHEVs) are rapidly gaining popularity on our roads. Besides a comparatively high purchasing price, the main two problems limiting their use are the short driving range and inconvenient charging process. In this paper we address the following by presenting an automatic robot-based charging station with 3D vision guidance for plugging and unplugging the charger. First of all, the whole system concept consisting of a 3D vision system, an UR10 robot and a charging station is presented. Then we show the shape-based matching methods used to successfully identify and get the exact pose of the charging port. The same approach is used to calibrate the camera-robot system by using just known structure of the connector plug and no additional markers. Finally, a three-step robot motion planning procedure for plug-in is presented and functionality is demonstrated in a series of successful experiments.

3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Veh...

Justas Miseikis

Mažasis Universitetas, IT karjera: Robotika

Justas Miseikis

CUDA based Iris Detection based on Hough Transform

Justas Miseikis

TESP 2012 Drums Haptic Interface

Justas Miseikis

Finger Rehabilitation Robot - Justinas Miseikis

Justas Miseikis

Mais de Justas Miseikis (8)

Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and ...

Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D P...

Automatic Calibration of a Robot Manipulator and Multi 3D Camera System

3D Vision Guided Robotic Charging Station for Electric and Plug-In Hybrid Veh...

Mažasis Universitetas, IT karjera: Robotika

CUDA based Iris Detection based on Hough Transform

TESP 2012 Drums Haptic Interface

Finger Rehabilitation Robot - Justinas Miseikis

Último

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Zilliz

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

Manulife - Insurer Transformation Award 2024

The Digital Insurer

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Scalable LLM APIs for AI and Generative AI Application Development Ettikan Karuppiah, Director/Technologist - NVIDIA Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...

apidays

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Whatsapp Number Escorts Call girls 8617370543 Available 24x7 Navi Mumbai Call Girls Service Offer Genuine VIP Model Escorts Call Girls in Your Budget. Navi Mumbai Call Girls Service Provide Real Call Girls Number. Make Your Sexual Pleasure Memorable with Our Navi Mumbai Call Girls at Affordable Price. Top VIP Escorts Call Girls, High Profile Independent Escorts Call Girls, Housewife Women Escorts Call Girl, College Girls Escorts Call Girls, Russian Escorts Call girls Service in Your Budget.

Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model

Deepika Singh

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Modernizing Securities Finance: The cloud-native prime brokerage platform transforming capital markets. Madhu Subbu, Managing Director, Head of Securities Finance Engineering Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu

apidays

Joint Human Detection from On-Board and Off-Board Cameras

1. Joint Human Detection from On-Board and Off-Board Cameras Justinas Mišeikis and Paulo V. K. Borges

2. Problem Various vehicles and people share the same environment. Unfortunately, in these conditions accidents occur.

3. Problem Can we use technology to prevent them?

4. Related Work • Pedestrian detection using vehicle mounted stereo cameras • Crowd tracking in enclosed environments • People tracking using ﬁxed cameras and driver warning if a person is on the path • Same object identiﬁcation using off-board cameras and not localized mobile camera

5. Our Method Overview Off-Board camera In world coordinates Pos relative to On-Board camera the vehicle detection

6. Our Method Overview

7. Off-Board Cameras • Fixed cameras • MOG2 background segmentation • Noise ﬁltering • Blob detection • Size and dimensions ﬁltering • Feet position - bottom centre point of the blob

8. People Detector - HOG • Popular method for people detection • Works well in cluttered environments • Based on distribution of intensity gradients or edge directions. • Descriptor created from many samples

9. On-Board Cameras Full Image Analysis The camera view is split according A: Whole Image analysis - person not to the distance from the camera: detected blue (3-7 meters), green (7-12 B: Area split method - person detected meters), yellow (12-20 meters).

10. Targeted Analysis Expected feet Detected position from person ground plane homography HOG search area Size calculated using the expected person height

11. Data Fusion Off-Board Cam A Off-Board Cam B Detected On-Board Cam - HOG people positions Position Fusion Final Pedestrian Position

12. Data Fusion Position sensor fusion 6 HOG detector variance Estimated distance error (meters) Measurement Error 5 Polynomial Error Estimation estimation depending 4 on the object’s distance 3 2 from the camera 1 0 2 3 4 5 6 7 8 9 10 11 12 Distance from the camera (meters) Extended Kalman Filter for sensor fusion and tracking

13. Implementation • C++ • Standard Intel laptop + desktop running in parallel • cvBlob library

14. Experiments 30m by 30m industrial site Test vehicle - HMC 2 Off-Board, 1 On-Board cam Three 4-6 minute runs 1-2 People walking around Vehicle static and moving

15. Results FP % FN % Localized Analysis 4.41 3.66 Full Image Analysis 95.53 2.40 • Works real-time • Background Segmentation - 10-15 FPS • HOG detectors - 5 FPS

16. Progress • Additional front facing cameras, 4 in total - Two side cameras - Wide angle camera for close proximity - Far proximity front camera • People tracking • HOG performed on GPU GeForce GT640

17. Progress Close Prox Cam Danger Zone - reduced to around 20 cm from the front of the vehicle Far Prox Cam

18. Thank You! Any Questions?

Joint Human Detection from On-Board and Off-Board Cameras

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Joint Human Detection from On-Board and Off-Board Cameras

Semelhante a Joint Human Detection from On-Board and Off-Board Cameras (20)

Mais de Justas Miseikis

Mais de Justas Miseikis (8)

Último

Último (20)

Joint Human Detection from On-Board and Off-Board Cameras

Notas do Editor