O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Safety and Validation of Autonomous Vehicles -- June 2019

323 visualizações

Publicada em

Safety and Validation of Autonomous Vehicles

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Safety and Validation of Autonomous Vehicles -- June 2019

  1. 1. Safety and Validation of Autonomous Vehicles June 26, 2019 © 2019 Edge Case Research Prof. Philip Koopman @PhilKoopman
  2. 2. 2© 2019 Edge Case Research  Perception safety approaches  Edge Cases as the limiting factor  The Heavy Tail Distribution Perception stress testing  Accelerate building the triggering event “zoo”  UL 4600: Autonomous Product Safety Standard  Transparent safety argument disclosure Overview [General Motors]
  3. 3. 3© 2019 Edge Case Research Validating an Autonomous Vehicle Pipeline Control Systems  Control Software Validation  Doer/Checker Architecture Autonomy Interface To Vehicle  Traditional Software Validation Perception presents a uniquely difficult assurance challenge Randomized & Heuristic Algorithms  Run-Time Safety Envelopes  Doer/Checker Architecture Machine Learning Based Approaches  ???
  4. 4. 4© 2019 Edge Case Research  Good for identifying “easy” cases  Expensive and potentially dangerous Brute Force AV Validation: Public Road Testing http://bit.ly/2toadfa
  5. 5. 5© 2019 Edge Case Research Validation Via Brute Force Road Testing?  If 100M miles/critical mishap…  Test 3x–10x longer than mishap rate  Need 1 Billion miles of testing  That’s ~25 round trips on every road in the world  With fewer than 10 critical mishaps …
  6. 6. 6© 2019 Edge Case Research  Safer, but expensive  Not scalable  Only tests things you have thought of! Closed Course Testing Volvo / Motor Trend
  7. 7. 7© 2019 Edge Case Research  Highly scalable; less expensive  Scalable; need to manage fidelity vs. cost  Only tests things you have thought of! Simulation http://bit.ly/2K5pQCN Udacity ANSYS
  8. 8. 8© 2019 Edge Case Research You should expect the extreme, weird, unusual  Unusual road obstacles  Extreme weather  Strange behaviors Edge Case are surprises  You won’t see these in testing  Edge cases are the stuff you didn’t think of! What About Edge Cases? https://www.clarifai.com/demo http://bit.ly/2In4rzj
  9. 9. 9© 2019 Edge Case Research  Novel objects (missing from zoo) are triggering events Need An Edge Case “Zoo” http://bit.ly/2top1KD http://bit.ly/2tvCCPK https://goo.gl/J3SSyu
  10. 10. 10© 2019 Edge Case Research  Where will you be after 1 Billion miles of fly-fix-fly?  Assume 1 Million miles between unsafe “surprises”  Example #1: 100 “surprises” @ 100M miles / surprise – All surprises seen about 10 times during testing – With luck, all bugs are fixed  Example #2: 100,000 “surprises” @ 100B miles / surprise – Only 1% of surprises seen during 1B mile testing – Bug fixes give no real improvement (1.01M miles / surprise) Why Edge Cases Matter https://goo.gl/3dzguf
  11. 11. 11© 2019 Edge Case Research Real World: Heavy Tail Distribution(?) Common Things Seen In Testing Edge Cases Not Seen In Testing (Heavy Tail Distribution)
  12. 12. 12© 2019 Edge Case Research  Need to find “Triggering Events” to inject into sims/testing The Heavy Tail Testing Ceiling
  13. 13. 13© 2019 Edge Case Research  Need to collect surprises  Novel objects  Novel operational conditions  Corner Cases vs. Edge Cases  Corner cases: infrequent combinations – Not all corner cases are edge cases  Edge cases: combinations that behave unexpectedly  Issue: novel for person ≠ novel for Machine Learning  ML can have “edges” in unexpected places  ML might train on features that seem irrelevant to people Edge Cases: Triggering Event Zoo https://goo.gl/Ni9HhU Not A Pedestrian
  14. 14. 14© 2019 Edge Case Research False negative when in front of dark vehicle False negative when person next to light pole Stress Testing Perception  Augmenting images with noise highlights perception issues  Identifies systemic weaknesses even in absence of noise False positive on lane marking False negative real bicyclist
  15. 15. 15© 2019 Edge Case Research  Brittle perception behavior indicates Edge Cases  Data augmentation reveals triggering events Automatically Detecting Edge Cases
  16. 16. 16© 2019 Edge Case Research  Mask-R CNN: examples of systemic problems Example Triggering Events via Hologram “Red objects” Notes: These are baseline, un-augmented images. (Your mileage may vary on your own trained neural network.) “Columns” “Camouflage” “Sun glare” “Bare legs” “Children” “Single Lane Control”
  17. 17. 17© 2019 Edge Case Research  Disengagements only catch near-collision surprises Going Beyond Disengagements
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. 21© 2019 Edge Case Research  Drivers do more than just drive  Occupant behavior, passenger safety  Detecting and managing equipment faults  Operational limitations & situations  System exits Operational Design Domain  Vehicle fire or catastrophic failure  Post-crash response  Interacting with non-drivers  Pedestrians, passengers  Police, emergency responders Operations & Human Interactions https://bit.ly/2GvDkUN https://bit.ly/2PhzilT
  22. 22. 22© 2019 Edge Case Research  Handling updates  Fully recertify after every weekly update?  Security in general  Vehicle maintenance  Pre-flight checks, cleaning  Corrective maintenance  Supply chain issues  Quality fade  Supply chain faults Lifecycle Issues https://bit.ly/2IKlZJ9 https://bit.ly/2VavsjM Is windshield cleaning fluid life critical?
  23. 23. 23© 2019 Edge Case Research  Safety case sufficiency  Hazards and risk mitigation  Development process  Machine learning and “AI”  Verification & Validation  Dependability & security  Data safety  Non-driver human interface  Metrics & assessment  Tool qualification  Maintenance  Other lifecycle concerns  Field data feedback  Dealing with unknowns UL 4600 Proposed Topics Include: UL4600.com
  24. 24. 24
  25. 25. 25© 2019 Edge Case Research  Unhelpful approaches:  “We’re really smart. Trust us.”  Buzzword roll calls  Silence  Show top pieces of a safety case  Why should the public believe?  Transparency is essential!  Include:  UL 4600-style safety argumentation  Credible independent assessment  Add: why your road testing is also safe Better AV Safety Disclosures
  26. 26. 26© 2019 Edge Case Research  More safety transparency  Independent safety assessments  Industry collaboration on safety  Minimum performance standards  “Driver test” is necessary -- but not sufficient – How do you measure maturity? – Accelerated perception testing  Autonomy software safety standards  ISO 26262/21448 + UL 4600 + IEEE P700x  Dealing with uncertainty and brittleness Ways To Improve AV Safety http://bit.ly/2MTbT8F (sign modified) Mars Thanks!

×