This document introduces Live Drone Map, a UAV-based rapid mapping and sharing solution. It consists of a UAV system that can automatically acquire geospatial data through sensors and process the data in real-time to generate orthoimages. The orthoimages are then uploaded and visualized in a cloud-based geospatial platform called Mago3D. This allows users to access and view mapping data updated in real-time from UAV flights through a web browser or mobile devices. The system is proposed to provide updated mapping for UN field operations by automating data acquisition, processing and sharing.
4. L o g o
What is a UAV?
UAVs: Unmanned Aerial Vehicles
“UAVs are to be understood as uninhabited and reusable
motorized aerial vehicles” (Blyenburg, 1999). These vehicles
are remotely controlled, semi-autonomous, autonomous, or
have a combination of these capabilities.
Main communities
Military, Artificial Intelligence, Computer Vision, Robotics,
Aeronautics, …
Geomatics (Photogrammetry, Remote Sensing and Surveying)
4
5. L o g o
UAV Mapping / Photogrammetry
UAV mapping / photogrammetry
opens various new applications in the close range domain,
combining aerial and terrestrial photogrammetry, but also
introduces low-cost alternatives to the classical manned aerial
photogrammtery.
In the context of mapping
Geospatial data collection with high geometric and temporal
resolution
(large scale data)
Generation of elevation models, orthophotos, maps, 3D
models etc.
5
6. L o g o
Motivation for the Use of UAVs
Advantages of UAVs
Use in high risk situations and inaccessible areas
Data acquisition with high temporal and spatial resolution
Autonomous and stabilized
Low-cost
Limitations in the use of UAVs
Limitations of the payload
Regulations and insurance
Use of Low-cost Sensors
6
7. L o g o
The accuracy of measurement methods in relation to the object/area size. Modified from
Fig. 1.4 in Luhmann, et al., 2006.
UAV Mapping vs. Others
7
8. L o g o
UAV Market Status and Prospects
Commercial drone market is growing due to
increasing demand of private sector
World commercial drone market size: Predicted to
grow up to 4 times larger (2016-2025)
Leisure: 2.2 Billion$ → 3.9 Billion$ - Large but saturated
Public: 36 Million$ → 464 Million$ – Small but growing fast
Commercial: 390 Million$ → 6.5 Billion$ – Biggest soon
8
9. L o g o
UAV Platform_DJI
DJI Drones
DJI’s products are intended for amateur as well as professional
use
Phantom series are the most popular product, and since
launch, have evolved to integrated flight programming with a
camera, WiFi connectivity, and the pilot’s mobile device.
9
10. L o g o
UAV Platform_Falcon8
Developed by Intel company
Automatic sensor data verification
3 flight modes for any circumstances
GPS mode, altitude mode, manual mode
Weight & Payload : 2.3kg / 0.8kg
Flight time : 16 min
10
11. L o g o
UAV Platform_Indago
Developed by Lockheed martin company
Flight time : 45 min (Maximum)
Sensor : Optical and infrared ray camera
Apply for Lifesave Project; find people who are
dementia patient
11
12. L o g o
UAV Platform_Skyranger
Developed by Aeryon company
Sustain extreme weather condition
Tolerate temperatures from 22 below zero to 122
degrees Faherenheit
Maneuver with wind gusts up to 55 miles per hour
Sensor HD camera(with infrared filters in low-light)
12
13. L o g o
UAV Platform_Sensefly
Developed by Parrot company
Cover up to 12km2 in a single automated flight
Weight : 500g
Sensors : image sensor, u-BLOX GPS chip, attitude
sensor, radio transmitter, autopilot circuit board
Flight time : 30 min
13
14. L o g o
Commercial Software
Software PhotoScan PhotoModeler Pix4D EnsoMOSAIC
Platform Linux, OS X,
Microsoft
Windows
Microsoft
Windows
Microsoft
Windows,
MacOS, Cloud
Microsoft
Windows
Automatic Yes Yes Yes Yes
Scalability Multiple
images
Multiple
images
Multiple
images
Multiple
images
Data source images images images images
Price $179~3499 $1145 $2000 $900
Online Service - - - -
14
15. L o g o
Opensource Software
Software MicMac OpenDroneM
ap
openMVG DroneMapper
Platform Linux, OS X,
Microsoft
Windows
Linux, OS X,
Microsoft
Windows
Linux, OS X,
Microsoft
Windows
Web-Based
Automatic Semi-
automatic
Yes Yes Yes
Scalability Multiple
images
Multiple
images
Multiple
images
Multiple
images
Data source images images images images
Price free free free free
Online Service Yes - - -
15
17. L o g o
DEM (Digital Elevation Model)
17
DEM(Digital Elevation Model)
is a digital 3D model created from
terrain elevation data.
can be represented as a grid or TIN
is often used as a generic term for
DSMs and DTMs.
DSM(Digital Surface Model)
represents the earth's surface and
includes all objects on it.
DTM(Digital Terrain Model)
represents the bare ground surface
without any objects like plants and
buildings.
GRID
TIN
18. L o g o
Orthoimage
18
Aerial images geometrically corrected (orthorectified)
such that the scale is uniform
adjusted for relief displacement, lens distortion, and camera
tilt.
overlapped with maps and used to measure true distances.
commonly used in GIS as a "map" background image.
(Row, Column, Color)
(Northing, Easting, Color)
+ Height
19. L o g o
3D Model
Represents terrain surfaces, sites, buildings, vegetation,
infrastructure and landscape elements as well as
related objects belonging to interested of areas.
19
Supports presentation,
exploration, analysis, and
management tasks in a large
number of different
application domains.
Allows "for visually
integrating heterogeneous
geoinformation within a
single framework”.
20. L o g o
Data Processing
20
Geo-
referncing
Dense
Matching
Meshing Texturing
Textured
3D
Model
EOPs
IOPs
Point
Clouds
Griding &
Interpolation
DSM
Rectification
Ortho-
image
21. L o g o
21
Live Drone Map2
UAV Based Rapid Mapping & Sharing Solution
22. L o g o
Background
UN has performed field operations such as
Peace keeping to monitor peace agreements
SAR(Search and Rescue) and restoring in disaster areas
Needs a GIS customized to a specific operation
The DPKO (Department of Peacekeeping Operation) develops
and implements field activities strategy based on the GIS.
The GIS should be established rapidly and applied to field
operations immediately.
The UNGSC (UN Global Service Centre) provides the mission
specific and adaptive GIS to the DPKO.
They used to employ existing sensory data such as satellite
images.
22
23. L o g o
Objectives
Usually, these existing images are out-of-date and
retain low resolution.
The mission areas may not have the existing data and
acquiring data newly takes time (4-6 weeks).
UAV mapping systems are very expeditious means to
acquire high resolution data on the mission areas.
We propose an automatic mapping system based on a
UAV, which can operate in a fully automatic way from
the data acquisition to the data processing.
23
24. L o g o
Role of UAV mapping system
The system is to provide the updated geo-spatial data
of a target area in a rapid and automatic way using a
UAV based multi-sensor system.
Input: the region of interests and the output options (types)
with the resolutions (1 cm~1 m)
Output: geo-spatial data such as orthoimage, DSM with a
user-specified resolution
24
25. L o g o
Key Advantages – Rapid & Automatic
A user just specifies the region of interests on a map
and the output options (types) with the required
resolutions.
The final product will be generated within less than
several hours (TBD) after the data acquisition.
All the remaining processes from the data acquisition
through data processing to final product generation
can be performed automatically.
The user does NOT require expert knowledge about
UAV flight planning and operation or
photogrammetric data processing.
25
26. L o g o
Concept of Operations
2. Data Acquisition
5. Geo-data Generation
3. Data Transfer
26
1. Preparation
4. Georeferencing6. Delivery
27. L o g o
Implementation Strategy
Plug-and-Play
We will develop a fully integrated multi-sensor payload
system by minimizing its dependency on a UAV platform.
The system does not heavily depends on a certain UAV
platform.
If certain requirements (max. payload weight) are met, we can
easily plug the payload system into the UAV and play its roles.
Hierarchical Modular Design
The system will be configured with a hierarchical modular
design approach.
Each module can be easily replaced for upgrade or
customization.
27
28. L o g o
Implementation Strategy
Low Cost / Open Source
Affordable commercial or low cost Open Source HW and SW
can be employed.
Hardware :
• UAV platforms: 3DR, DJI, DIY Kits, etc.
• Sensor control & integration: Arduino, Raspberry Pi
Software : Customized from Open-Source
• Flight control & planning
• Photogrammetric Data processing: Photoscan / OpenDroneMap
28
29. L o g o
UN Geo-Cloud Arch.- Data Collection
29
Proposed by Prof. K. Lee, Oct/01/2015
Secure and High Performance Storage
Data Collector
Applications
Metadata Imagery Data
Geo-Portal
Server
Vector Data Feature Data
1. UAV Collector
3. Map
Generalization
Field Collector
(Mil. Observer)
Trajectory
Collector
Geo-MM
Collector
2. Crowdsourcing Collector
Processing
Server
Feature
ServerImage
Server
GeoMM
Server
4. 3D
Collector
30. L o g o
Configuration of UAV Collector
Flight Planning Data Quality Check
Georeferencing Orthorectification
30
UAV Mapping System
UAV Platform
SensorsSensor Support
32. L o g o
Platforms
32
UAV(TAM GD 14000) UAV(D5) UAV (Q-TILT)
Model
A: Current Version
B: Upgrade by 2017
Spec.
Flight time
~60 minutes
(Average 35 minutes)
Flight time ~30min Flight time
A: 30 minutes
B: >60 minutes
Maximum
Payload
6,000g
Maximum
Payload
10kg Maximum Payload
A: 1,000g
B: > 2,000g
Speed
Rate of climb : 7.5 m/s
Cruising speed : 12.0
m/s
Speed 18m/s Speed
Rate of climb : 7.0
m/s
Cruising speed :
25.0 m/s
Operating
radius
1,000 m Wind limit 15m/s
Operating
radius
A: 5,000 m
B: >15,000 m
Weight 8,650g Weight 35kg Weight
A: 12,000 g
B: 20,000 g
Dimensions
140cm X 140cm X
70cm
Battery Size 44,000mAh / 44V Dimensions
140cm X 140cm X
70cm
Temperatur
e
-10°C ~ 50°C Temperature -10°C ~ 50°C
34. L o g o
Sensors
34
Digital Camera (& Lens) GPS/INS Lidar
Model
Sony A7 II SONY Sonnar T*
FE 35mm F2.8 ZA
APX-15
(Applanix single board GNSS-Inertial solution)
Velodyne
Spec.
Resolutions 6000x4000
- SBAS
- Support PPS time synchronization
16 channel
Pixel size 5.97um Operating Temp 40° ~ 75° Measurement
Range : ~100m
ISO 100~25600 Dimensions 67 x 60 x 15mm
Maximum Shutter
speed
1/8000sec Weight 60g ±3cm Accuracy
Weight(Camera) 599g Accuracy SPS DGPS RTK4
Post-
Processe
d5 FOV : ±15
Degree (Vertical)
360 Degree
(Horizontal)
Focal length 35mm Position(m) 1.5-3.0 0.5-2.0 0.02-0.05 0.02-0.05
Size 61.5x36.5mm
Velocity(m/s) 0.05 0.05 0.02 0.015
Roll&Pitch(deg) 0.04 0.03 0.03 0.025
Wavelenth :
903nm
Weight(Lens) 120g True Heading3(deg) 0.30 0.28 0.18 0.080
Single and Dual
Returns
35. L o g o
Sensor Control & Data Transmission
To control sensors, collect the sensory data (images)
with time tags and transmit them to the ground.
Based on open source HW such as Latte (single board
computer) & long-range WiFi
Time accuracy: < 10ms
35
Latte (single board computer)
Long-range WiFi
36. L o g o
Data Processing SW
36
Produce individual orthoimages and mosaics of
mission areas automatically and rapidly
PA Data
Data
Processing
SW
Individually
Geo-rectified
Images
Images
Ref.
Data
Mosaic
Orthoimage
DSM
Rapidly &
Automatically
UAV Pos/Att from GPS/INS
from Cameras
Existing Geo-data
(GCPs, DEMs, Ortho-images)
Real-time Data Processing
in several seconds after each image acquisition
Rapid Post-Processing
in several hours after a flight mission
37. L o g o
SW at GitHub
Real-time Data Processing
A subset for Live Drone Map
Download sensory data
Process them to generate individual orthoimage
Upload the orthoimage to a cloud server (Mago3D)
https://github.com/flyhamsw/UN_symposium_v1.2
Rapid Post-processing
Georeferencing SW
https://github.com/flyhamsw/Georeferencing
Ortho-mosaic Generation SW
https://github.com/flyhamsw/Orthophoto-generation
37
40. L o g o
Live Drone Map
40
UAV Based Real-time Mapping & Sharing Solution
We combine the real-time mapping solution (UAV
collector) with a cloud based geo-data sharing
solution (Mago3D).
Users in distant areas with Internet can access the
image maps updated in real-time using UAV systems
during its flight.
The image maps are visualized in 2D/3D with existing
geo-data without any plug-in software through a
standard web-browser on a desktop computer or even
mobile devices.
Smartphone images can be uploaded and shared.
41. L o g o
Visualization in Live Drone Map
41
3D on a Desktop Computer 2D on a Smartphone
42. L o g o
Applications of Live Drone Map
42
2. Real-time Data Collection 4. All-source Situation Room
1. Disaster
Occurance
UAV + Sensor payload(camera+GPS/INS)
Real-time transmission
of images and Pos./Att.
Real-time transmission
of imagery maps
3. Real-time Mapping
43. L o g o
1st Demonstration in Korea
43
In Seoul on Nov. 9, 2016.
3rd International Symposium
Partnership for Technology in Peacekeeping
Information and Communications Technology Division
United Nations Department of Field Support
Indoor demonstration
inside Seoul City Hall
45. L o g o
2nd Demonstration in Korea
45
In Seoul on Feb. 21, 2017.
46. L o g o
2nd Demonstration in Korea
Digital Times
https://www.youtube.com/watch?v=yTrdPNbn_LY&feature=you
tu.be
YTN news
http://m.ytn.co.kr/news_view.php?key=201702212159482174&s
_mcd=0102
46
47. L o g o
Integrated with MCC on Apr. 27, 2017.
3rd Demonstration at UN GSC in Italy
47
Mission Area
(GIS team/patrol)
MCC
Mission HQ
(Commander)
San Pancrazio Airfield
UN GCS, Brindisi
New York 40 km
> 10 hour via flight
48. L o g o
3rd Demonstration: Configuration
48
Reception
& Archiving
Processing
Acquisition &
Transmission
Geo-rectified
Image
Sensory
Data
UAV Collector
Updating
(Geo-portal)
Delivery /
Visualization
(Users)
Mago3D
Smartphone
Crowd sourcing
MCC
53. L o g o
3rd Demonstration : Day 1 (Apr. 20)
53
시험, 시연, 교육, 워크숍 등 세부 일정 협의
무인기 비행허가 및 장비 배송 확인
시연 시나리오 확인
라이브 드론맵 개발과정, 무인기 매핑 워크숍 일정 확인
Air Field로 이동하여 실험 진행
지상 SW 실험 : 기존 데이터를 저장소에 전송하여 영상처리를
진행하고 지오포털 상에 가시화 되는 것을 확인
무인기 비행을 제외하고 실내에서 센서 탑재체의 데이터가 WiFi
통신을 통하여 저장소로 전송되는 것 확인
문제점 확인
이탈리아 현지의 통신 상태가 불안정하여 핸드폰 테더링을 통해
LTE 통신 이용
54. L o g o
3rd Demonstration : Day 1 (Apr. 20)
54
55. L o g o
3rd Demonstration : Day 2 (Apr. 21)
55
지상 SW 점검
테더링 스마트폰을 이용하여 네트워크 설정
GSD를 변경하며 업로드 시간 체크
센서 탑재체 실외시험 진행
긴 멀티텝을 사용하여 실외에서 센서 데이터가 저장소로 전송되
는 것을 확인
MCC와의 협의
MCC : 통신과 전력이 고립된 지역에서도 운영이 가능하도록 설
계된 컨테이너 박스 형태의 시스템
MCC 내부에 있는 컴퓨터를 이용하여 클라우드 서버 접속 후 갱
신되는 지도 확인
GIS 부서 견학
56. L o g o
3rd Demonstration : Day 2 (Apr. 21)
56
57. L o g o
3rd Demonstration : Day 3 (Apr. 24)
57
무인기 비행 테스트 진행(Apr. 24)
이착륙 및 1차 수동 비행 시험
Waypoint 경로 설정 후 2차 자율 비행 시험
탑재체 장착 후 3차 자율 비행 시험
지상 SW 점검
기존 데이터를 저장소에 전송하여 영상처리를 진행하고 지오포
털 상에 가시화 되는 것을 확인
워크숍 진행
사진측량 및 무인기 매핑 사례 소개
라이브 드론맵의 개발과정 및 구성과 동작원리
58. L o g o
3rd Demonstration : Day 3 (Apr. 24)
58
59. L o g o
3rd Demonstration : Day 4 (Apr. 25)
59
1차 통합 비행 시험
AR Works에서 보유한 무인기 2대 비행시험
• 이탈리아 현지 바람이 너무 강하게 불어 크기가 작은 무인기를 이용
하여 시연하기로 결정
Apx data의 문제점 확인 : 보드 문제
MCC와의 통신 확인
UN 실무자 견학
무인기 매핑 실습 진행
WebODM 설치 및 실행과정 설명
Client 서버 상에 설치된 WebODM에 데이터를 전송하여 영상
처리를 진행하고 결과(3D point cloud, 정사영상) 확인
60. L o g o
3rd Demonstration : Day 4 (Apr. 25)
60
61. L o g o
3rd Demonstration : Day 5 (Apr. 26)
61
지상 SW 점검
지상 SW와 MCC 통합 시험
MCC 인터넷망(위성 기반)의 속도를 고려하여 2D 환경에서 4초
마다 영상을 갱신하기로 결정
센서 탑재체 확인 및 문제점 체크
무인기 정류장치 대신 외장 배터리를 이용하여 전원 공급
통합 비행 시험
MCC 팀과 연계하여 2차 통합 비행 시험
62. L o g o
3rd Demonstration : Day 5 (Apr. 26)
62
64. L o g o
3rd Demonstration : Day 6 (Apr. 27)
64
San Pancrazio Airfield 현장에서
무인기 이륙
영상 촬영 후 지상으로 전송
지상 SW에서 지오레퍼런싱 수행 후
클라우드 서버로 전송
MCC에서 위성 인터넷으로 클라우드
서버에 접속, Mago3D를 통해 실시간
으로 가시화
66. L o g o
3rd Demonstration : Day 6 (Apr. 27)
66
67. L o g o
3rd Demonstration : Day 6 (Apr. 27)
67
68. L o g o
Conclusions
Live Drone Map can acquire multi-sensory data and
produce high resolution geospatial information in real-
time and visualize them rapidly.
With Live Drone Map, we can quickly acquire high
resolution image maps of mission areas with no maps
or old maps only.
The accuracy of the image maps are about 1~5 m
depending on the flight altitude without any GCP.
These latest maps will be useful for many important
tasks in UN field operations, such as monitoring
disputed areas and restoring disaster areas.
68
69. L o g o
Future Work
We need to apply our Live Drone Map to real mission
areas such as Lebanon and improve it for more
practical use.
We are currently using WiFi and LTE networks, but we
will employ RF links and mobile networks of the
mission areas.
We are currently processing data on the computers in
mission areas, but we will also perform the data
processing on a cloud server.
We will test with cheaper drones such as DJI ones,
which are widely used in the field.
69