"Making Cozmo See," a Presentation from Anki

•

6 gostaram•2,696 visualizações

For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-stein For more information about embedded vision, please visit: http://www.embedded-vision.com Andrew Stein, Lead Computer Vision Engineer at Anki, presents the "Making Cozmo See" tutorial at the May 2017 Embedded Vision Summit. In this presentation, Stein describes the vision capabilities of Cozmo, Anki's latest consumer robotics product. Cozmo is a sophisticated entertainment robot focused on personality, interactivity, and game play. It was one of the hottest toys of 2016, earning a spot amongst the top two best-selling premium toys of the holiday season. In addition to covering high-level architecture and functionality, Stein discusses some of the challenges of building a capable vision system at a scale and price point for the consumer market.

Tecnologia

Copyright © 2017 Anki, Inc. 1
Andrew Stein
May 2017
Making Cozmo See

Copyright © 2017 Anki, Inc. 2
Introducing Anki & Cozmo
• About Anki [http://www.anki.com]
• Consumer Robotics & AI Company, currently focused
on entertainment
• Two Products: Overdrive and Cozmo
• About Cozmo [http://www.anki.com/en-us/cozmo]
• App-driven, autonomous robot character
• Launched Fall 2016, $179
• Full-featured SDK (low- and high-level API)

Copyright © 2017 Anki, Inc. 3
• Primary Goal: Bring Cozmo to Life
• All about personality, via animation
• Interactive, aware
About Cozmo

Copyright © 2017 Anki, Inc. 4
Cozmo’s Personality

Copyright © 2017 Anki, Inc. 5
• Outputs:
• 4 Mechanical DoF: Head, Lift, Treaded Wheels
• Speaker
• OLED face
• Sensors:
• Camera (VGA, 15fps)
• Cliff sensor
• Encoders and IMU
• Battery Life: 1-2 hours
• 3 Processors: comms, sensors/motor control, supervisor
Cozmo’s Core Hardware Capabilities

Copyright © 2017 Anki, Inc. 6
Simple Linear Flow
Script Animation Library
I wish!

Copyright © 2017 Anki, Inc. 7
Cozmo Flow
Reaction Triggers
Game
Logic
World Model
Path Planning
Sensors
Emotional
State
Controlled
Movement
Animation Library
Behaviors

Copyright © 2017 Anki, Inc. 8
• Cozmo moves
• Motion blur
• Must not drive off cliffs
• Odometry not reliable (treads!)
• Cozmo gets moved
• “Robot Kidnapping” not just
academic with six-year-olds!
• How to maintain and
communicate world state?
• Cozmo interacts: Faces, cubes, charger, etc.
Not a Static Surveillance Camera

Copyright © 2017 Anki, Inc. 9
Vision Capabilities

Copyright © 2017 Anki, Inc. 10
• Detect/track/recognize: cubes, charger, faces, motion, etc.
• Robust enrollment for face recognition
• Make eye contact
• Estimate 3D pose of objects
• Maintain map of the world (even if picked up)
• Rolling shutter correction
• Flexible scheduling according to needs and computational budget
Vision Capabilities

Copyright © 2017 Anki, Inc. 11
• Store up to 10 identities across play sessions, enrolled via app activity
• Remember 3D poses for eye contact
Face Detection, Tracking, & Recognition

Copyright © 2017 Anki, Inc. 12
Cat & Dog Face Detection

Copyright © 2017 Anki, Inc. 13
Markers: Cubes, Charger, & Custom Objects (SDK)
Custom Markers for SDK

Copyright © 2017 Anki, Inc. 15
Markers: Design vs. Vision
Vs.

Copyright © 2017 Anki, Inc. 16
Detecting “Interesting Stuff”

Copyright © 2017 Anki, Inc. 17
System Architecture

Copyright © 2017 Anki, Inc. 18
• Use Smart Device for heavy
computation
• Reduce robot cost
• Better development environment
System Architecture
UI + Game Logic
Robotics
Engine
(Vision / AI)
Hardware Drivers
Low-Level
Controllers
Comms
Python
SDK
CLAD via Wifi CLAD via USB
CLAD: Custom
Cross-Platform Messaging
(similar to protobufs)
CLAD (Local)

Copyright © 2017 Anki, Inc. 19
• Animations don’t play instantaneously (must buffer audio)
• Face Enrollment in “Meet Cozmo” activity (limited visual feedback)
• State Mirroring (from engine to Robot / Game / SDK)
• Latency for vision and control
Architecture Complications

Copyright © 2017 Anki, Inc. 21
Under the Hood of the Full System

Copyright © 2017 Anki, Inc. 22
Under the Hood (Manual Drive)

Copyright © 2017 Anki, Inc. 23
Under the Hood (Autonomous)

Copyright © 2017 Anki, Inc. 24
Computer Vision in a Product at Scale

Copyright © 2017 Anki, Inc. 25
Computer Vision is Not the Product
Realistic
Expectations
Product
Requirements
Computer Vision

Copyright © 2017 Anki, Inc. 26
Computer Vision at Scale: Calibration
Manipulating cubes requires high precision
• Camera variability (lens placement)
• Mechanical tolerances
Must calibrate each camera at factory
• Verify camera placement inside head
• Estimate intrinsic parameters

Copyright © 2017 Anki, Inc. 27
• Cannot control
lighting in
people’s homes
• Cube lights
complicate local
binarization
Computer Vision at Scale: Illumination

Copyright © 2017 Anki, Inc. 28
• Detecting and responding to failures
is as important as being successful
Failure Detection

Copyright © 2017 Anki, Inc. 30
• Cozmo: interactive, complex, full of personality
• Computer vision is only a piece of the product,
does not drive all decisions
• Computer vision at scale, in a consumer product,
comes with special challenges and considerations
• Setting expectations for what is (not) possible with
vision is important in a world of sci-fi TV & movies
• Cozmo is a fun little robot to develop for
Conclusions

Copyright © 2017 Anki, Inc. 31
What Cozmo is All About

Copyright © 2017 Anki, Inc. 32
Join us!
We’re
hiring!

Mais conteúdo relacionado

Mais de Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/computer-vision-in-sports-scalable-solutions-for-downmarkets-a-presentation-from-sportlogiq/ Mehrsan Javan, Co-founder and CTO of Sportlogiq, presents the “Computer Vision in Sports: Scalable Solutions for Downmarket Leagues” tutorial at the May 2023 Embedded Vision Summit. Sports analytics is about observing, understanding and describing the game in an intelligent manner. In practice, this requires a fully automated, robust end-to-end pipeline, spanning from visual input, to player and group activities, to player and team evaluation to planning. Despite major advancements in computer vision and machine learning, today sports analytics solutions are limited to top leagues and are not widely available for downmarket leagues and youth sports. In this talk, Javan explains how his company has developed scalable and robust computer vision solutions to democratize sport analytics and offer pro-league-level insights to leagues with modest resources, including youth leagues. He highlights key challenges—such as the requirement for low-cost, low-latency processing and the need for robustness despite variations in venues. He discusses the approaches Sportlogiq tried and how it ultimately overcame these challenges, including the use of transformers and fusion of multiple type of data streams to maximize accuracy.

“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/detecting-data-drift-in-image-classification-neural-networks-a-presentation-from-southern-illinois-university/ Spyros Tragoudas, Professor and School Director at Southern Illinois University Carbondale, presents the “Detecting Data Drift in Image Classification Neural Networks” tutorial at the May 2023 Embedded Vision Summit. An unforeseen change in the input data is called “drift,” and may impact the accuracy of machine learning models. In this talk, Tragoudas presents a novel scheme for diagnosing data drift in the input streams of image classification neural networks. His proposed method for drift detection and quantification uses a threshold dictionary for the prediction probabilities of each class in the neural network model. The method is applicable to any drift type in images such as noise and weather effects, among others. Tragoudas shares experimental results on various data sets, drift types and neural network models to show that his proposed method estimates the drift magnitude with high accuracy, especially when the level of drift significantly impacts the model’s performance.

“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/deep-neural-network-training-diagnosing-problems-and-implementing-solutions-a-presentation-from-sensor-cortek/ Fahed Hassanat, Chief Operating Officer and Head of Engineering at Sensor Cortek, presents the “Deep Neural Network Training: Diagnosing Problems and Implementing Solutions” tutorial at the May 2023 Embedded Vision Summit. In this presentation, Hassanat delves into some of the most common problems that arise when training deep neural networks. He provides a brief overview of essential training metrics, including accuracy, precision, false positives, false negatives and F1 score. Hassanat then explores training challenges that arise from problems with hyperparameters, inappropriately sized models, inadequate models, poor-quality datasets, imbalances within training datasets and mismatches between training and testing datasets. To help detect and diagnose training problems, he also covers techniques such as understanding performance curves, recognizing overfitting and underfitting, analyzing confusion matrices and identifying class interaction issues.

“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/ai-start-ups-the-perils-of-fishing-for-whales-war-stories-from-the-entrepreneurial-front-lines-a-presentation-from-seechange-technologies/ Tim Hartley, Vice President of Product for SeeChange Technologies, presents the “AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepreneurial Front Lines)” tutorial at the May 2023 Embedded Vision Summit. You have a killer idea that will change the world. You’ve thought through product-market fit and differentiation. You have seed funding and a world-beating team. Best of all, you’ve caught the attention of major players in your industry. You’ve reached peak “start-up”—that point of limitless possibility—when you go to bed with the same level of energy and enthusiasm you had when you woke. And then the first proof of concept starts… In this talk, Hartley lays out some of the pitfalls that await those building the next big thing. Using real examples, he shares some of the dos and don’ts, particularly when dealing with that big potential first customer. Hartley discusses the importance of end-to-end design, ensuring your product solves real-world problems. He explores how far the big companies will tell you to jump—and then jump again—for free. And, most importantly, how to build long-term partnerships with major corporations without relying on over-promising sales pitches.

“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/a-computer-vision-system-for-autonomous-satellite-maneuvering-a-presentation-from-scout-space/ Andrew Harris, Spacecraft Systems Engineer at SCOUT Space, presents the “Developing a Computer Vision System for Autonomous Satellite Maneuvering” tutorial at the May 2023 Embedded Vision Summit. Computer vision systems for mobile autonomous machines experience a wide variety of real-world conditions and inputs that can be challenging to capture accurately in training datasets. Few autonomous systems experience more challenging conditions than those in orbit. In this talk, Harris describes how SCOUT Space has designed and trained satellite vision systems using dynamic and physically informed synthetic image datasets. Harris describes how his company generates synthetic data for this challenging environment and how it leverages new real-world data to improve our datasets. In particular, he explains how these synthetic datasets account for and can replicate real sources of noise and error in the orbital environment, and how his company supplements them with in-space data from the first SCOUT-Vision system, which has been in orbit since 2021.

“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/bias-in-computer-vision-its-bigger-than-facial-recognition-a-presentation-from-santa-clara-university/ Susan Kennedy, Assistant Professor of Philosophy at Santa Clara University, presents the “Bias in Computer Vision—It’s Bigger Than Facial Recognition!” tutorial at the May 2023 Embedded Vision Summit. As AI is increasingly integrated into various industries, concerns about its potential to reproduce or exacerbate bias have become widespread. While the use of AI holds the promise of reducing bias, it can also have unintended consequences, particularly in high-stakes computer vision applications such as facial recognition. However, even seemingly low-stakes computer vision applications such as identifying potholes and damaged roads can also present ethical challenges related to bias. This talk explores how bias in computer vision often poses an ethical challenge, regardless of the stakes involved. Kennedy discusses the limitations of technical solutions aimed at mitigating bias, and why “bias-free” AI may not be achievable. Instead, she focuses on the importance of adopting a “bias-aware” approach to responsible AI design and explores strategies that can be employed to achieve this.

“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/sensor-fusion-techniques-for-accurate-perception-of-objects-in-the-environment-a-presentation-from-sanborn-map-company/ Baharak Soltanian, Vice President of Research and Development for the Sanborn Map Company, presents the “Sensor Fusion Techniques for Accurate Perception of Objects in the Environment” tutorial at the May 2023 Embedded Vision Summit. Increasingly, perceptual AI is being used to enable devices and systems to obtain accurate estimates of object locations, speeds and trajectories. In demanding applications, this is often best done using a heterogeneous combination of sensors (e.g., vision, radar, LiDAR). In this talk, Soltanian introduces techniques for combining data from multiple sensors to obtain accurate information about objects in the environment. Soltanian briefly introduces the roles played by Kalman filters, particle filters, Bayesian networks and neural networks in this type of fusion. She then examines alternative fusion architectures, such as centralized and decentralized approaches, to better understand the trade-offs associated with different approaches to sensor fusion as used to enhance the ability of machines to understand their environment.

“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/updating-the-edge-ml-development-process-a-presentation-from-samsara/ Jim Steele, Vice President of Embedded Software at Samsara, presents the “Updating the Edge ML Development Process” tutorial at the May 2023 Embedded Vision Summit. Samsara (NYSE:IOT) is focused on digitizing the world of operations. The company helps customers across many industries—including food and beverage, utilities and energy, field services and government—get information about their physical operations into the cloud, so they can operate more safely, efficiently and sustainably. Samsara’s sensors collect billions of data points per day and on-device processing is instrumental to its success. The company is constantly developing, improving and deploying ML models at the edge. Samsara has found that the traditional development process—where ML scientists create models and hand them off to firmware engineers for embedded implementation—is slow and often produces difficult-to-resolve differences between the original model and the embedded implementation. In this talk, Steele presents an alternative development process that his company has adopted with good results. In this process, firmware engineers develop a general framework that ML scientists use to design, develop and deploy their models. This enables quick iterations and fewer confounding bugs.

“Updating the Edge ML Development Process,” a Presentation from Samsara

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/combating-bias-in-production-computer-vision-systems-a-presentation-from-red-cell-partners/ Alex Thaman, Chief Architect at Red Cell Partners, presents the “Combating Bias in Production Computer Vision Systems” tutorial at the May 2023 Embedded Vision Summit. Bias is a critical challenge in predictive and generative AI that involves images of humans. People have a variety of body shapes, skin tones and other features that can be challenging to represent completely in training data. Without attention to bias risks, ML systems have the potential to treat people unfairly, and even to make humans more likely to do so. In this talk, Thaman examines the ways in which bias can arise in visual AI systems. He shares techniques for detecting bias and strategies for minimizing it in production AI systems.

“Combating Bias in Production Computer Vision Systems,” a Presentation from R...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/developing-an-embedded-vision-ai-powered-fitness-system-a-presentation-from-peloton-interactive/ Sanjay Nichani, Vice President for Artificial Intelligence and Computer Vision at Peloton Interactive, presents the “Developing an Embedded Vision AI-powered Fitness System” tutorial at the May 2023 Embedded Vision Summit. The Guide is Peloton’s first strength-training product that runs on a physical device and also the first that uses AI technology. It turns any TV into an interactive personal training studio. Members have access to Peloton’s instructors, who lead classes that use dumbbells and body weight. The Guide uses an innovative combination of hardware, software and user interface; it is one of only a handful of consumer products in this market that use real-time embedded computer vision, and Peloton’s members love it! In this talk, Nichani shares insights into how the computer vision technology behind the Guide was built. He focuses on data, models and processes. The AI and Computer Vision team at Peloton worked through numerous obstacles by using synthetic data, advanced algorithms and field-testing iteration to create an affordable AI-powered device that enables at-home fitness for everyone.

“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/navigating-the-evolving-venture-capital-landscape-for-edge-ai-start-ups-a-presentation-from-hpe-pathfinder/ Todd Poole, Director of Venture Capital Investments for HPE Pathfinder Ventures, presents the “Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups” tutorial at the May 2023 Embedded Vision Summit. The venture capital environment has undergone a huge shift in the past year. In this talk, Poole explains how the VC landscape has changed and the implications for start-ups. He examines the impacts on valuations, VC’s willingness to fund, appetite for risk and leverage in deal structuring and terms negotiation. Pool explores how AI and ML hardware and software companies have been affected differently from the broader start-up ecosystem and considers the distinct challenges faced by start-ups in the edge AI sector. He offers guidance for founders, soon-to-be-founders and start-up employees on how to navigate this dynamic environment and strategize effectively in response to changing market conditions.

“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/advanced-presence-sensing-what-it-means-for-the-smart-home-a-presentation-from-omdia/ Jack Narcotta, Principal Analyst for the Smart Home at Omdia, presents the “Advanced Presence Sensing: What It Means for the Smart Home” tutorial at the May 2023 Embedded Vision Summit. Eventually, homes will become highly autonomous—powered by ubiquitous, connected, intelligent devices (sometimes referred to as ambient computing)—but this remains a distant vision. In the meantime, advanced presence sensing innovations from component manufacturers, software developers and smart home device companies are enabling meaningful improvements in convenience, efficiency and safety. Omdia expects advanced presence detection to be adopted rapidly in the coming years, and to become a key factor in how consumers select devices and service providers. In this talk, Narcotta provides an overview of key enabling technologies, market appetite, consumer expectations and long-term outlook for advanced presence sensing in the home.

“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/tracking-and-fusing-diverse-risk-factors-to-drive-a-safer-future-a-presentation-from-nauto/ Yoav Banin, Chief Product and Business Development Officer, and Tahmida Mahmud, Engineering Manager for Perception, both of Nauto, presents the “Tracking and Fusing Diverse Risk Factors to Drive a SAFER Future” tutorial at the May 2023 Embedded Vision Summit. Unless you’re a gang member or drug addict, driving is your top risk. But which risks can you handle and which can kill you? Since 96% of collisions are caused by human error and 75% are unconscious (the driver is not aware of the risk), Nauto used one billion miles of real-life driving data to find out, feeding it into a model fusing 26 road context, driver action/attention and vehicle dynamics factors to predict collisions, near misses and safe driving. Nauto, as described by Banin and Mahmud in this presentation, found that drivers can handle most single risks, but multiple simultaneous risks can be deadly. Some combinations are 1,000 times more dangerous than regular driving. Nauto’s “SAFER” model can be fitted to any late-model vehicle. It can be trained with labeled data from historical trips or through reinforcement learning from observations of excellent and poor drivers. A time series matrix can help you see a few seconds into the future.

“Tracking and Fusing Diverse Risk Factors to Drive a SAFER Future,” a Present...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/mipi-csi-2-image-sensor-interface-standard-features-enable-efficient-embedded-vision-systems-a-presentation-from-the-mipi-alliance/ Haran Thanigasalam, Camera and Imaging Consultant to the MIPI Alliance, presents the “MIPI CSI-2 Image Sensor Interface Standard Features Enable Efficient Embedded Vision Systems” tutorial at the May 2023 Embedded Vision Summit. As computer vision applications continue to evolve rapidly, there’s a growing need for a smarter standardized interface connecting multiple image sensors to processors for real-time perception and decision-making. In this presentation, Thanigasalam provides a deep dive into the latest version of the widely implemented CSI-2 v4.0 interface from MIPI Alliance. This new version includes key features specifically designed to support computer vision applications, including democratized Smart Region of Interest, Always-On Sentinel Conduit, Multi-Pixel Compression and Latency Reduction and Transport Efficiency. These novel features enable sophisticated machine awareness with reduced system power and processing needs, making them well suited for consumer, commercial and infrastructure platforms.

“MIPI CSI-2 Image Sensor Interface Standard Features Enable Efficient Embedde...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/introduction-to-the-csi-2-image-sensor-interface-standard-a-presentation-from-the-mipi-alliance/ Haran Thanigasalam, Camera and Imaging Consultant to the MIPI Alliance, presents the “Introduction to the MIPI CSI-2 Image Sensor Interface Standard” tutorial at the May 2023 Embedded Vision Summit. By taking advantage of select features in standardized interfaces, vision system architects can help reduce processor load, cost and power consumption while gaining flexibility to source components from multiple vendors. In this presentation, Thanigasalam introduces the standardized MIPI CSI-2 imaging conduit solution for interfacing image sensors to processors. Thanigasalam explains how the solution supports basic operations of a camera sensor, including physical frame transport options and orthogonal commands for autoexposure, autofocus, auto white balance and event detection. He also introduces command sets to help bring up and tune sensors. In addition, Thanigasalam explores provisions to alleviate RF emissions, enable aggregation of data from multiple remote sensors and alleviate the need for dual voltage signaling to avoid electrical overstress issues. He also touches on emerging support for frame transport using WiFi.

“Introduction to the CSI-2 Image Sensor Interface Standard,” a Presentation f...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/practical-approaches-to-dnn-quantization-a-presentation-from-magic-leap/ Dwith Chenna, Senior Embedded DSP Engineer for Computer Vision at Magic Leap, presents the “Practical Approaches to DNN Quantization” tutorial at the May 2023 Embedded Vision Summit. Convolutional neural networks, widely used in computer vision tasks, require substantial computation and memory resources, making it challenging to run these models on resource-constrained devices. Quantization involves modifying CNNs to use smaller data types (e.g., switching from 32-bit floating-point values to 8-bit integer values). Quantization is an effective way to reduce the computation and memory bandwidth requirements of these models, and their memory footprints, making it easier to run them on edge devices. However, quantization does degrade the accuracy of CNNs. In this talk, Chenna surveys practical techniques for CNN quantization and shares best practices, tools and recipes to enable you to get the best results from quantization, including ways to minimize accuracy loss.

“Practical Approaches to DNN Quantization,” a Presentation from Magic Leap

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/optimizing-image-quality-and-stereo-depth-at-the-edge-a-presentation-from-john-deere/ Travis Davis, Delivery Manager in the Automation and Autonomy Core, and Tarik Loukili, Technical Lead for Automation and Autonomy Applications, both of John Deere, present the “Reinventing Smart Cities with Computer Vision” tutorial at the May 2023 Embedded Vision Summit. John Deere uses machine learning and computer vision (including stereo vision) for challenging outdoor applications such as obstacle detection, vision-based guidance and weed management, among many others. The quality of the images the company’s systems obtain, and the accuracy of the depth information produced by its stereo cameras, significantly impact the performance of the overall solutions. In this talk, Davis and Loukili share some of the challenges John Deere has faced in developing image quality improvement and stereo vision algorithms. Many of the techniques found in academic research and prior work cannot be easily implemented in real-time applications at the edge and are difficult to scale for applications with varying performance and cost requirements. They highlight some of the alternative techniques their company has developed to provide optimized image-quality and stereo vision implementations that meet the requirements of John Deere’s product range. Stated another way, they share how they do more with less.

"Optimizing Image Quality and Stereo Depth at the Edge," a Presentation from ...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/using-a-collaborative-network-of-distributed-cameras-for-object-tracking-a-presentation-from-invision-ai/ Samuel Örn, Team Lead and Senior Machine Learning and Computer Vision Engineer at Invision AI, presents the “Using a Collaborative Network of Distributed Cameras for Object Tracking” tutorial at the May 2023 Embedded Vision Summit. Using multiple fixed cameras to track objects requires a careful solution design. To enable scaling the number of cameras, the solution must avoid sending all images across a network. When camera views overlap only a little or not at all, input from multiple cameras must be combined by extending tracking area coverage. When an object can be seen by multiple cameras, this should be used to increase accuracy. Multiple cameras can better collaborate if they share a common coordinate system; therefore, environment mapping and accurate calibration is necessary. Moreover, the tracking algorithm must scale properly with the number of tracked objects, which can be achieved with a distributed approach. In this talk, Örn covers practical ways of addressing these issues, presents his company’s multiple-camera tracking solution used for vehicle and pedestrian tracking, and shares some of its results.

“Using a Collaborative Network of Distributed Cameras for Object Tracking,” a...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/a-survey-of-model-compression-methods-a-presentation-from-instrumental/ Rustem Feyzkhanov, Staff Machine Learning Engineer at Instrumental, presents the “Survey of Model Compression Methods” tutorial at the May 2023 Embedded Vision Summit. One of the main challenges when deploying computer vision models to the edge is optimizing the model for speed, memory and energy consumption. In this talk, Feyzkhanov provides a comprehensive survey of model compression approaches, which are crucial for harnessing the full potential of deep learning models on edge devices. Feyzkhanov explores pruning, weight clustering and knowledge distillation, explaining how these techniques work and how to use them effectively. He also examines inference frameworks, including ONNX, TFLite and OpenVINO. Feyzkhanov discusses how these frameworks support model compression and explores the impact of hardware considerations on the choice of framework. He concludes with a comparison of the techniques presented, considering implementation complexity and typical efficiency gains.

“A Survey of Model Compression Methods,” a Presentation from Instrumental

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/09/reinventing-smart-cities-with-computer-vision-a-presentation-from-hayden-ai/ Vaibhav Ghadiok, Co-founder and CTO of Hayden AI, presents the “Reinventing Smart Cities with Computer Vision” tutorial at the May 2023 Embedded Vision Summit. Hayden AI has developed the first AI-powered data platform for smart and safe city applications such as traffic enforcement, parking and asset management. In this talk, Ghadiok presents his company’s privacy-preserving vision-based mobile solution, which it deploys in a city’s existing transportation fleet to collect real-time data. Hayden AI’s mobile intelligent camera detects objects in the environment, such as vehicles, pedestrians and license plates, while the company’s location engine fuses data from multiple sensors (such as cameras, GNSS and IMUs) to locate these objects in 3D. Ghadiok shares some of the key business and technical challenges his company has faced in developing and deploying this solution with partners like New York City’s Metropolitan Transportation Authority, and how it has overcome these challenges. He also explains how these cameras, deployed across a fleet of vehicles, can collaboratively upload data to build a real-time 3D re-creation of a city, providing insights that can be used to improve safety and efficiency.

“Reinventing Smart Cities with Computer Vision,” a Presentation from Hayden AI

Edge AI and Vision Alliance

Mais de Edge AI and Vision Alliance (20)