O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

PDR Challenge in Warehouse Picking and Virtual Mapping Party

556 visualizações

Publicada em

This talk consists of two parts. One is a report on the
progress of PDR Challenge in Warehouse Picking which will be held as a
competition track in IPIN 2017, Sapporo, Japan. Another is a brief
introduction of Virtual Mapping Party which enables the participants to
contribute to the accessible information collection for visually
impaired people from anywhere and at any time.

Publicada em: Engenharia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

PDR Challenge in Warehouse Picking and Virtual Mapping Party

  1. 1. 国立研究開発法人 PDR Challenge in Warehouse Picking and Virtual Mapping Party Takeshi Kurata12 1AIST, 2Univ. of Tsukuba
  2. 2. 国立研究開発法人 PDR(Pedestrian Dead-Reckoning) Estimates velocity vector, relative altitude, and action type by measurements from a wearable sensor module.  Wearing a sensor module on waist (2D SHS (Steps and Heading Systems) PDR)  Easy to wear and maintain  Easy to measure data for action recognition  Relatively easily apply for handheld setting compared to shoe-mounted PDR (3D-INS (Inertial Navigation System) PDR) 2 Handheld PDR From PDR to PDRplus 10-axis sensors • Accelerometers • Magnetic sensors • Gyro sensors • Barometer Shoe-mounted PDR Waist-worn PDR
  3. 3. 国立研究開発法人 ISMAR2003 Image registration + PDR PLANS2010 PDRplus (PDR + Action recognition) PLANS2014 Handheld PDR (Walking Direction Estimation) 2015- 2015- PDR Module ICAT2006 PDR + GPS + RFID G-spatial EXPO2010 PDR demo on iPhone 4 ICServ2013 Best Paper: Computer-Supported QC Circle (CSQCC) with PDR IPIN2015, 2016 10-axes PDR, Time-series sub-optimization in SDF 2011- Docomo map navi (As of Sep, 2016, 480 areas including subways and underground shopping arcades in Japan) 2014- 産総研技術移転ベンチャー PDR Benchmark Standardization Committee Established in 2014 (Chair: T. Kurata, 33 organizations in Japan as of April, 2016) Overview: History of PDR in AIST AIST Start-up AcademiaIndustry PDR: Pedestrian Dead Reckoning SDF: Sensor Data Fusion (Hybrid Positioning) RFID: Radio Frequency Identifier GPS: Global Positioning System
  4. 4. 国立研究開発法人 AR by PDR + Image registration (1999-2003) Panorama-based Annotation: IWAR1999, ISWC2001, ISMAR2003 G Environmental map A B C D E A B C F Input frames Position at which a panorama is taken Position Direction 235 [deg] 5 [deg] From the user’s camera Located Orientated 4
  5. 5. 国立研究開発法人 ISMAR2003 Image registration + PDR PLANS2010 PDRplus (PDR + Action recognition) PLANS2014 Handheld PDR (Walking Direction Estimation) 2015- 2015- PDR Module ICAT2006 PDR + GPS + RFID G-spatial EXPO2010 PDR demo on iPhone 4 ICServ2013 Best Paper: Computer-Supported QC Circle (CSQCC) with PDR IPIN2015, 2016 10-axes PDR, Time-series sub-optimization in SDF 2011- Docomo map navi (As of Sep, 2016, 480 areas including subways and underground shopping arcades in Japan) 2014- 産総研技術移転ベンチャー PDR Benchmark Standardization Committee Established in 2014 (Chair: T. Kurata, 33 organizations in Japan as of April, 2016) Overview: History of PDR in AIST AIST Start-up AcademiaIndustry PDR: Pedestrian Dead Reckoning SDF: Sensor Data Fusion (Hybrid Positioning) RFID: Radio Frequency Identifier GPS: Global Positioning System
  6. 6. 国立研究開発法人 In the year of 2010 • iPhone 4: the first popular consumer mobile device equipped with 9-axis sensors including accelerometers, magnetic sensors, and gyro sensors 6 G-spatial EXPO 2010: Handheld PDR on iPhone 4 (Worldʼs first-ever live demo) PLANS2010, PLANS2014
  7. 7. 国立研究開発法人 ISMAR2003 Image registration + PDR PLANS2010 PDRplus (PDR + Action recognition) PLANS2014 Handheld PDR (Walking Direction Estimation) 2015- 2015- PDR Module ICAT2006 PDR + GPS + RFID G-spatial EXPO2010 PDR demo on iPhone 4 ICServ2013 Best Paper: Computer-Supported QC Circle (CSQCC) with PDR IPIN2015, 2016 10-axes PDR, Time-series sub-optimization in SDF 2011- Docomo map navi (As of Sep, 2016, 480 areas including subways and underground shopping arcades in Japan) 2014- 産総研技術移転ベンチャー PDR Benchmark Standardization Committee Established in 2014 (Chair: T. Kurata, 33 organizations in Japan as of April, 2016) Overview: History of PDR in AIST AIST Start-up AcademiaIndustry PDR: Pedestrian Dead Reckoning SDF: Sensor Data Fusion (Hybrid Positioning) RFID: Radio Frequency Identifier GPS: Global Positioning System
  8. 8. 国立研究開発法人 Frontier of PDR: Walking direction estimation 8 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.
  9. 9. 国立研究開発法人 Frontier of PDR: Walking direction estimation 9 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015. • Long Paper: Christophe Combettes, Valerie Renaudin, Comparison of Misalignment Estimation Techniques Between Handheld Device and Walking Directions, IPIN 2015. • FIS was proposed by Kourogi and Kurata in PLANS 2014. “Globally, the FIS method provides better results than the other two methods.” by IFSTTAR Frequency analysis of Inertial Signals Forward and Lateral Acc. Modeling Principal Component Analysis
  10. 10. 国立研究開発法人 ISMAR2003 Image registration + PDR PLANS2010 PDRplus (PDR + Action recognition) PLANS2014 Handheld PDR (Walking Direction Estimation) 2015- 2015- PDR Module ICAT2006 PDR + GPS + RFID G-spatial EXPO2010 PDR demo on iPhone 4 ICServ2013 Best Paper: Computer-Supported QC Circle (CSQCC) with PDR IPIN2015, 2016 10-axes PDR, Time-series sub-optimization in SDF 2011- Docomo map navi (As of Sep, 2016, 480 areas including subways and underground shopping arcades in Japan) 2014- 産総研技術移転ベンチャー PDR Benchmark Standardization Committee Established in 2014 (Chair: T. Kurata, 33 organizations in Japan as of April, 2016) Overview: History of PDR in AIST AIST Start-up AcademiaIndustry PDR: Pedestrian Dead Reckoning SDF: Sensor Data Fusion (Hybrid Positioning) RFID: Radio Frequency Identifier GPS: Global Positioning System
  11. 11. 国立研究開発法人 Global Trend on PDR PDR R&D players have rapidly indicated their presence all over the world on and after 2010. Movea (France) Sensor Platforms (USA) CSR (UK) TRX (USA) Trusted Positioning (Canada) 11 Acquired by QualcommAcquired by InvenSenseAcquired by InvenSense Acquired by Audience Indoo.rs (USA) SFO Acquired by TDK?
  12. 12. 国立研究開発法人 ISMAR2003 Image registration + PDR PLANS2010 PDRplus (PDR + Action recognition) PLANS2014 Handheld PDR (Walking Direction Estimation) 2015- 2015- PDR Module ICAT2006 PDR + GPS + RFID G-spatial EXPO2010 PDR demo on iPhone 4 ICServ2013 Best Paper: Computer-Supported QC Circle (CSQCC) with PDR IPIN2015, 2016 10-axes PDR, Time-series sub-optimization in SDF 2011- Docomo map navi (As of Sep, 2016, 480 areas including subways and underground shopping arcades in Japan) 2014- 産総研技術移転ベンチャー PDR Benchmark Standardization Committee Established in 2014 (Chair: T. Kurata, 33 organizations in Japan as of April, 2016) Overview: History of PDR in AIST AIST Start-up AcademiaIndustry PDR: Pedestrian Dead Reckoning SDF: Sensor Data Fusion (Hybrid Positioning) RFID: Radio Frequency Identifier GPS: Global Positioning System
  13. 13. 国立研究開発法人 Standardization on PDR Benchmarking • PDR related R&D is highly active worldwide: Necessity for sharing common measures. • Description of the performance should be unified in spec sheets and scientific papers. • Different measures from absolute positioning methods such as GNSS, Wi-Fi, and BLE are required for PDR, which is a method of relative positioning. • PDR Benchmark Standardization Committee was established in 2014 as a platform of the grassroots activity. 13https://www.facebook.com/pdr.bms
  14. 14. 国立研究開発法人
  15. 15. 国立研究開発法人 Scene in data collection 15
  16. 16. 国立研究開発法人 Multi-Algorithm On-Site Evaluation System • Evaluates the accuracy of each PDR algorithm automatically as often as sensor data is uploaded to the server • Provides trajectory images so that participants can compare their PDR • algorithms in real time. 16 http://pdrsv.hasc.jp K. Kaji, K. Kanagu, K. Murao, N. Nishio, K. Urano, H. Iida, N. Kawaguch, Multi-Algorithm On-Site Evaluation System for PDR Challenge, ICMU2016.
  17. 17. 国立研究開発法人 UbiComp/ISWC 2015 PDR Challenge Corpus • Is now open to the public. (http://hub.hasc.jp/) 17 Routes 5 Devices 7 Subjects 93 # of pedestrian sensing data 241 # of pedestrian sensing data with calibration data 230 # of pedestrian sensing data with LIDAR data 10 Avg. of walking time [sec] 101 Avg. of moving distance [m] 115 Avg. of angular change [°] 606 K. Kaji, M. Abe, W. Wang, K. Hiroi, and N. Kawaguchi, UbiComp/ISWC 2015 PDR challenge corpus, HASCA2016 (UbiComp2016 Proceedings: Adjunct), pp.696-704 Statistics of the corpus Detailed route statistics of pedestrian sensing data with calibration data
  18. 18. 国立研究開発法人 Open Data Contest in Logistics & PDR Challenge in Warehouse Picking • Open data contest in logistics by Frameworx – Submission: 2016/4/18- 2016/7/18 – Award ceremony: 2016/9/12 • PDR Challenge in Warehouse Picking – Will be held as an international contest in IPIN 2017 18
  19. 19. 国立研究開発法人 PDR Challenge Series • Ubicomp/ISWC 2015 PDR Challenge – Scenario: Indoor Navigation – On-site – Continuous walking while keeping watching the navigation screen by holding the smartphone – Several minutes per trial • 2017 PDR Challenge in Warehouse Picking – Scenario: Picking work in a warehouse – Off-site – Not only walking but various actions including picking and carrying – Several hours per trial – Will be held in IPIN 2017 19
  20. 20. 国立研究開発法人 Examples of picking workersʼ trajectories estimated by PDR + WMS (Warehouse Management System) 20
  21. 21. 国立研究開発法人 How to design benchmark Indicators? • Other aspects to be considered – Reliability: Different measures from absolute positioning methods are required for PDR – Efficiency: Power consumption – Repeatability: Temperature Hysteresis, Magnetic field, etc. – Representativeness: How to hold, Route shape, etc. 21 Benchmark indicators of vision-based spatial registration and tracking for MAR (ISO/IEC WD 18520) (PEVO: Projection Error of Virtual Objects)
  22. 22. 国立研究開発法人 How to compare and visualize? 22 Easy Difficult Method 1 Easy Difficult Method 2
  23. 23. 国立研究開発法人 How to compare and visualize? 23 Easy Difficult Method1Method2
  24. 24. 国立研究開発法人 Competitions: IPIN and the others (cf. EvAAL presentation in IPIN 2105 etc.) 24 IPIN year EvAAL, IPSN, UbiComp/ISWC Zurich, Switzerland 2010 universAAL is launched Guimaraes, Portugal 2011 EvAAL: indoor localization Sidney, Australia 2012 EvAAL: + activity recognition Montbeliard, France 2013 EvAAL: same as 2012 Busan, Korea 1st IPIN competition 2014 EvAAL: 3 floors, smartphone IPSN: infrastruc. based + free Banff, Canada EvAAL-ETRI comp. 2015 EvAAL-ETRI: 6 floors, on/off-site IPSN: infrastruc. based + free UbiComp/ISWC: 2 floors, smartphone PDR, 90 subjects Madrid, Spain Indoor Localization Competition 2016 IPIN: smartphone (on/off-site), PDR, Robot IPSN: infrastruc. based + free, 2D/3D
  25. 25. 国立研究開発法人 IPIN2017 25
  26. 26. 国立研究開発法人 Virtual Mapping Party 26 which enables the participants to contribute to the accessible information collection for visually impaired people from anywhere and at any time.
  27. 27. 国立研究開発法人 Characteristics of each mapping work 27 Type of activities location Time Remarks Conventional mapping party On-site Sync. Face to face communication Deep understanding of real conditions Mandatory skill for organizing events Up to weather Mapping party utilizing smartphones app. On-site Any time (Async.) Mapping while commuting Easy to contribute Deep understanding of real conditions Position of contents depending on localization methods Mapping party utilizing crowdsourcing image sharing service Anywhere (Off-site) Any time (Async.) Crowdsourcing Remote mapping Easy to contribute anytime and anywhere Depend on shared data Limited understanding of real conditions Virtual mapping party Anywhere (Off-site) Any time (Async.) Crowdsourcing Remote mapping Easy to contribute anytime and anywhere Easy to measure contentsʼ position Easy to verify registered contents Mandatory pre-recording
  28. 28. 国立研究開発法人 How to decide POI/POR position 28 Using intersection of line of sight and the ground Triangulating with plural panoramas. POI: Point Of Interest (Landmark such as Store, restaurant, hospital, facilities, etc.) POR: Point Of Reference (specific point location the existence of which is easily recognized for confirming routes such as characteristic shape and material of ground (steps, stairs, sloop, door), sound/noise, and scent/odor.)
  29. 29. 国立研究開発法人 Screenshots for the virtual mapping interface. 29
  30. 30. 国立研究開発法人 Desktop vs. Smartphone VR 30 • The number of registered POI/PORs in WSs held at Miraikan : 598 (42 participant, 6 one-hour Workshops) • Sense of Immersion: Desktop << Cardboard HMD • Registration efficiency: Desktop/Smartphone=1.43 POI/POR/Request on OSM
  31. 31. 国立研究開発法人 Feedback from WS participants 31 Categories of feedback Positive feedback Negative feedback/ Suggestions About VR experience with omnidirectional images/movies and 3D sound I like the function for 3D sounds. 3D sounds seem to be very useful, since visually impaired people can confirm amount of traffic on roads. I could more realistically experience the VR scene by omnidirectional movies than still omnidirectional images. Estimating direction of sound sources was difficult. The quality of the images/movies was not perfect. About devices for VR experience I like instant HMD, since we can experience VR with what I have. I like VR experience with Oculus VR HMD since I can realistically experience by movies. I like Samsung's Gear VR HMD, because the image quality looked good and it was confortable for wearing. It took a while to get used to HMD, and I got tired when I wore HMD. Mapping with PC is better in terms of degree of fatigue. An instant HMD was not so comfortable for wearing. I thought wearing HMD on glass was difficult. About user interfaces I like the function for pointing in first personʼs view not map view. PC is the easiest platform for inputting POR/POI. About AR Tactile map I like the function for sending request by visually impaired people. The accuracy of gesture recognition for AR tactile map needs to be improved. About POR/POIs There are so many POR/POIs in the display. I think it becomes more clear if the displayed contents are limited to nearby contents. I found empty POR/POIs without detailed information. I think filtering of the registered contents are required. Other suggestions I wondered if the system could support communication between participants. I would like to regularly contribute virtual mapping parties from my home. It was the most beneficial application of VR I have ever experienced.
  32. 32. 国立研究開発法人 Usage of the AR tactile map for virtual mapping party • Allowing the visually-impaired people to join the mapping party by gesture – Search: Confirming POR/POI on the tactile map with sound for telling existence of POR/POI where user touches – Tap: Confirming POR/POI with Text-to-Sound when user taps the specific point – Double Tap: Requesting the POR/POI registration for specific points of the map
  33. 33. 国立研究開発法人 AR tactile maps with HP Sprout
  34. 34. 国立研究開発法人 Automatic identification and tracking of tactile maps 34 • ORB Feature point detector/local feature descriptor is used for identifying tactile map with RGB image • Estimating homography matrix between rectified image templates and input image

×