Mais conteúdo relacionado
Semelhante a ROS Ecosystem & Applications [HyphaROS] -- by HaoChih (20)
ROS Ecosystem & Applications [HyphaROS] -- by HaoChih
- 3. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
About us
● Introduction
○ Website: https://hypharosworkshop.wordpress.com/
○ Github: https://github.com/Hypha-ROS/hypha-racecar
○ Facebook: https://www.facebook.com/HyphaROS/
○ Youtube: https://www.youtube.com/watch?v=jgS6gpmWPoY
○ Youku: http://i.youku.com/hypha
- 4. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
About us
● Introduction
○ Established date: April-2017
○ Self-organized Workshop: 3
○ Speech & Training: 6
○ Technical supports: > 10 universities/labs & > 5 companies
○ HyphaROS RaceCar ver 1.0 released (sold 30 units)
4
- 5. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
About us
● Introduction
○ Lecturer in “ROS Summer School in China 2017”
○ Presenter in “2017 WRC ROS Forum”
○ Product & Service [ROS]: Workshop, Technical Consultant, AGVs, Home robots, etc..
○ Cooperate with Chips Company (China), will attend CES 2018 (if everything goes right...)
- 18. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
ROS Introduction
● Package
○ include/package_name: C++ include headers
○ msg/: Folder containing Message (msg) types
○ src/: Source files
○ srv/: Folder containing Service (srv) types
○ scripts/: executable scripts
○ CMakeLists.txt: CMake build file (see catkin/CMakeLists.txt)
○ package.xml: Package catkin/package.xml
● Node
● Message [A message is simply a data structure, comprising typed fields (much like C structs).]
● Master [provides name registration and lookup to the rest of the Computation Graph]
● Parameter Server
● Topics
● Service [for request / reply interactions]
● Rosbag
● Actionlib [A common, topic-based interface for preemptible tasks in ROS.]
● Plugins [provides a library for dynamically loading libraries in C++ code.]
- 19. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
ROS Introduction
● [concept] Establishing a topic connection
○ Subscriber starts.
○ Publisher starts.
○ Subscriber registers with the Master. (XMLRPC)
○ Publisher registers with the Master. (XMLRPC)
○ Master informs Subscriber of new Publisher. (XMLRPC)
○ Subscriber contacts Publisher to request a topic
connection and negotiate the transport protocol. (XMLRPC)
○ Publisher sends Subscriber the settings
for the selected transport protocol. (XMLRPC)
○ Subscriber connects to Publisher using
the selected transport protocol. (TCPROS, etc...)
figure source: http://wiki.ros.org/ROS/Technical%20Overview
- 22. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
ROS Introduction
• Standard Linux tools:
Compilers, Debuggers, Loggers, IDEs
• Multiple language support:
C/C++, Python, Java, Lisp, Go, Scratch, etc.
• Standard libraries:
Boost, MySQL, XML (whatever you can imagine)
• Modern GUI tools
• Many tools missing from industrial development packages
Source: ROS-Industrial Introduction slides 2013, Shaun Edwards
- 47. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
ROS Introduction
Source: https://software.intel.com/en-us/articles/build-an-autonomous-mobile-robot-with-the-intel-realsense-camera-ros-and-sawr
IntelROSProject
Realsense
Movidius
- 54. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
• Website
1. ROS Install (Kinetic): http://wiki.ros.org/kinetic/Installation/Ubuntu
2. ROS Wiki (Tutorial): http://wiki.ros.org/ROS/Tutorials
3. CLEARPATH Tutorial: http://www.clearpathrobotics.com/assets/guides/ros/
• Youtube Video
1. ROS Tutorial:
https://www.youtube.com/playlist?list=PLSzYQGCXRW1HLWHdJ7ehZPA-nn7R9UKPa
2. Erle Robotics:
https://www.youtube.com/playlist?list=PL39WpgKDjDfVfiNVG47DBi93wsh2XHKVO
3. The Construct: http://www.theconstructsim.com/
ROS Introduction
- 56. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
● Gmapping
○ needs external odom (in our case: laser odom fused with imu0)
○ Rao-Blackwellized Particle Filter (RBPF)
○ Loop closure
● Hector SLAM
○ Gauss-Newton approach
○ External odom is not necessary
○ No loop closure
○ Requires high quality sensor (e.g. UTM-30LX)
○ Supports EKF fusion (imu)
● MRPT-ICP SLAM
○ ICP algorithm
○ For small to mid-sized maps
○ External odom is not necessary
2D Laser SLAM
- 60. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
● 3D Laser
○ google cartographer
○ ETHZ-ASL libpointmatcher
○ ETHZ-ASL laser_slam
● Visual (RGBD)
○ Rtabmap
○ RGBD SLAM (v1, v2)
● Visual Stereo
○ Duo-VIO
● Visual Monocular
○ PTAM
○ LSD_SLAM (semi-dense)
○ SVO (semi-dense)
○ ORB-SLAM (feature based)
● Visual Inertial Odometry/SLAM
○ Loosely coupled: MSF_EKF
○ Tightly coupled: OKVIS, ROVIO, Duo-VIO
○ Framework: ETHZ-ASL maplab
3D SLAM & Odometry
Source: https://github.com/ethz-asl/maplab
- 63. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
3D SLAM & Odometry
Tutorials
1.RGB-D Hand-Held Mapping
2.This tutorial shows how to use rtabmap_ros out-of-the-box with a Kinect-like sensor in mapping mode or localization mode.
3.Stereo Hand-Held Mapping
4.This tutorial shows how to use rtabmap_ros out-of-the-box with a stereo camera in mapping mode or localization mode.
5.Remote Mapping
6.This tutorial shows how to do mapping on a remote computer.
7.Stereo Outdoor Mapping
8.This tutorial shows how to do stereo mapping with RTAB-Map.
9.Stereo Outdoor Navigation
10.This tutorial shows how to integrate autonomous navigation with RTAB-Map in context of outdoor stereo mapping.
11.Setup RTAB-Map on Your Robot!
12.This tutorial shows multiple RTAB-Map configurations that can be used on your robot.
13.Mapping and Navigation with Turtlebot
14.This tutorial shows how to use RTAB-Map with Turtlebot for mapping and navigation.
15.Tango ROS Streamer
16.Tutorial to get Tango ROS Streamer working with rtabmap_ros
17.Advanced Parameter TuningThis tutorial tells you which parameter to change to improve performances
18.Wifi Signal Strength Mapping (User Data Usage)
19.This tutorial shows how to add user data during mapping that will be saved directly in RTAB-Map's database for convenience.
● RtabMap Real-Time Appearance-Based Mapping
- 65. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
3D SLAM & Odometry
● RtabMap Real-Time Appearance-Based Mapping
● 3D mapping on Odroid XU4 with Realsense R200
ORB vs GFTT/ORB (BRISK is not good enough)
320 * 240
loop closure (g2o)
Vis + ICP
PCL filter and octomap
The IMU issue (mpu6050 -> gy85)
Supports multi-mapping
- 68. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
3D SLAM & Odometry
● SVO 1 & 2 http://rpg.ifi.uzh.ch/svo2.html
1. Parrot-SenseFly Albris drone:
https://www.youtube.com/watch?v=mYKrR...
2. Virtual reality:
https://www.youtube.com/watch?v=k0MLs...
3. iPhone app 3D around:
https://3daround.dacuda.com/
4. iPhone app Staramba3D:
https://itunes.apple.com/us/app/stara...
5. Zurich-Eye (now Facebook-Oculus VR):
http://www.zurich-eye.com/
- 70. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
● ETHZ-ASL libpointmatcher
3D SLAM & Odometry
Source: https://github.com/ethz-asl/libpointmatcher
An "Iterative Closest Point" library for 2-D/3-D mapping in Robotics
- 74. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Introduction
● The requirements of the platform
○ ROS fully support
○ Able to implement 2D Navigation stack (laser based)
○ Low cost (ARM SBC, low-cost lidar, mems imu, etc)
○ High speed/performance
○ Robust & Safety
○ Algorithms evaluation/comparison
○ Modularization & Extensibility
- 75. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Introduction
● The origin of RACECAR
MIT: https://mit-racecar.github.io/
Youtube: https://www.youtube.com/watch?v=9fzzp6oxid4
1/10-scale Traxxis Rally Car
Nvidia Jetson TX 1
Hokuyo UST-10LX laser range finder
Stereolabs ZED stereo camera
Structure.io depth camera
Sparkfun IMU
Encoder + optical flow
realtime onboard 4 m/s
Total cost: around 3,600 USD
- 76. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Introduction
● Similar projects
Penn: http://f1tenth.org/ [without slam, NAV]
UCB: http://www.barc-project.com/projects/ [without laser]
Georgia Tech: https://github.com/AutoRally [for outdoor]
CMU: https://github.com/mrsd16teamd/loco_car [indoor]
- 79. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Introduction
● Our Solution
○ Open-Source, Open-Hardware
○ Low-cost (~ 600 USD), High-speed (~ 3 m/s)
○ Step by step tutorial for beginner (available on github: Hypha-ROS)
Version 1 Version 2
- 82. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Hardware Architecture
Odroid XU4
Arduino
Nano
USB
Hub
Gy85
Servo
Motor ESC
Arduino
UNO
RPLidar
(A1M8)
FTDI
USB
USB
I2C
PWM
PWM
Wifi
TX/RX
USB
Ethernet
7.4V BatteyBEC
DC 2 DC
Core
Motor
BEC
- 83. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Software Architecture
move_base
(path planning)
robot_localization
(EKF)
rf2o
(laser odom)
PySerial
(imu reading)
Filtered
Odometry • Pure Pursuit
• TEB control
• MPC
rosserial
(to arduino)
rplidar_ros
(laser scan)
Ackerman
(rpm odom)
GPS
(optional)
- 85. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
HyphaROS RaceCar: Github
HyphaROS
VM image
Desktop sharing
(vino + remmina)
Terminal or File
manager (ssh)
All source code can be found
on Hypha-ROS github:
https://github.com/Hypha-
ROS/hypha-racecar
(including udev setting, Arduino
files, necessary documents)
- 87. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Drone Applications
● Simulation
○ Gazebo (with PX4)
○ AirSim Simulation
● Well known Projects
○ APM
○ PX4 & Dronecode
○ PaparazziUAV
○ CrazySwarm
○ Erlerobot gym-gazebo
○ DJI SDK
○ ARDrone SDK
- 92. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
MCU
(Pixhawk, APM, Arduino)SBC (Odroid XU4, Nvidia TX1/2 or NUC)
Drone Applications
● General Architecture (Loosely coupled)
MoveIt
(octomap &
Path)
Trajectory
Generator
(optional)
Serial
e.g. MAVROS
PySerial
VO/SLAM
Sensors
Imu, gps
MSF-
EKF
RGB
camera
High level
Controller
Platform
(HW)
Low level
Controller
- 96. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
ARDrone Indoor Navigation
LSD-SLAM
PTAM (IMU)
Least
square
MoveIt
(octomap & RRT*)
Trajectory
Generator
NDI
Controller
In: RGB image
In:
• RGB image
• IMU
• Sonar
Out: scaled pcl Out: way points Out: trajectory
Out: cmd
Could be
replaced by
MSF-EKF
- 97. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
●LSD-SLAM (Large-Scale Direct Monocular SLAM)
As direct method, LSD-SLAM uses all information in the image, including
e.g. edges – while keypoint-based approaches can only use small
patches around corners. This leads to higher accuracy and more
robustness in sparsely textured environments (e.g. indoors), and a much
denser 3D reconstruction.
ARDrone Indoor Navigation
- 99. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
𝑀 𝑞𝑙 𝑖
𝛿𝑞 − 𝑞𝑡 𝑖
2
Am+1 = Am + 𝑀 𝑇 𝑞𝑙 𝑚+1
𝑀(𝑞𝑙 𝑚+1
)
ym+1 = ym + 𝑀 𝑇 𝑞𝑙 𝑚+1
𝑞𝑡 𝑚+1
[𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑖
𝛿𝑥
𝛼
− 𝑥 𝑡 𝑖
|2
Am+1 = Am + 𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
𝑇
𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
ym+1 = ym + 𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
𝑇
𝑥 𝑡𝑖
Recursive least-squares
M is quaternion multiplication matrix
R is quaternion rotation matrix
ARDrone Indoor Navigation
Monocular SLAM non scaled issue
- 100. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Let 𝑝 𝑛, denote the coefficients of a polynomial P of degree N such that
P t = 𝑃𝑛 𝑡 𝑁
+ 𝑃𝑛−1 𝑡 𝑛−1
. . . +𝑝0 =
𝑛=0
𝑁
𝑃𝑛 𝑡 𝑛
We are interested in optimizing the coefficients of P to minimize cost functions (lateral offset
from a nominal path) of the form
J =
0
𝜏
𝑐0 𝑃 𝑡 2
+ 𝑐1 𝑃′
𝑡 2
+ 𝑐2 𝑃′′
𝑡 2
+ . . . +𝑐 𝑁 𝑃 𝑁
𝑡 2
𝑑𝑡
which can be written in quadratic form as, J = 𝑝 𝑇
𝑄 𝑝
where
Q =
𝑟=0
𝑁
𝑐 𝑟 𝑄 𝑟
Qr
𝑖𝑙
=
2(
𝑚=0
𝑟−1
(𝑖 − 𝑚)(𝑙 − 𝑚))
𝜏 𝑖+𝑙−2𝑟+1
𝑖 + 𝑙 − 2𝑟 + 1
; 𝑖 ≥ 𝑟 ∧ 𝑙 ≥ 𝑟
0; 𝑖 < 𝑟 ∨ 𝑙 < 𝑟
Control, Estimation, and Planning Algorithms for Aggressive Flight using Onboard Sensing, MIT, Adam Parker Bry
ARDrone Indoor Navigation
Trajectory Generating
- 101. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Piecewise Polynomial Joint Optimization
We now have a quadratic program (QP) of the form: min
𝑥
𝑥 𝑇
𝑄𝑥
s. t. Ax − b = 0
Since we wish to "correct" for derivative discontinuities in Dubins curves it is necessary
to jointly optimize multiple polynomial segments
Control, Estimation, and Planning Algorithms for Aggressive Flight using Onboard Sensing, MIT, Adam Parker Bry
ARDrone Indoor Navigation
Trajectory Generating
- 102. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Piecewise Polynomial Joint Optimization
The higher order time differential constraints for curve-segment of Dubins curves:
ARDrone Indoor Navigation
Trajectory Generating
- 103. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Piecewise Polynomial Joint Optimization
The higher order time differential constraints for line-segment of Dubins curves:
ARDrone Indoor Navigation
Trajectory Generating
- 104. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
𝑥 = 𝑥 𝑤 𝑦 𝑤 zw 𝑢 𝑤 𝑣 𝑤 𝑞1 𝑞2 𝑞3 𝑞4
𝑇
𝑢 = 𝜙 𝜃 𝜓 𝑧 = [𝑢1 𝑢2 𝑢3 𝑢4]
𝑉 = 𝑧 𝑤 = 𝑢4
𝑢1 = tan−1
𝑥
𝑔 − 𝑧
𝑢2 = tan−1
𝑦𝑐𝑜𝑠𝑢1
𝑔 − 𝑧
𝑢3 = 𝜓
ARDrone Indoor Navigation
Nonlinear Dynamic Inversion (NDI)
- 109. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Source: Source: ROS-Industrial slides 2017, 邱强 (上海交大)
Inverse
Kinematics
(KDL, IK-FAST)
Collision
Checking
(FCL)
Motion
Planning
(OMPL: RRT)
Forward
Kinematics
(URDF, TF)
ROS Industrial
- 116. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Robot Arms
• Motoman
• Adept
• Universal
• Fanuc
• ABB
• Kuka
• Techman (Taiwan)
• More to come…
Source: ROS-Industrial Introduction slides 2013, Shaun Edwards
ROS Industrial
- 122. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
●New Horizon 2020 robotics projects from 2016
https://ec.europa.eu/programmes/horizon2020/en/news/new-horizon-
2020-robotics-projects-2016
The two biggest projects are ROS related !
RoBMoSys and ROSIN (15.5 MEUR in total)
http://rosin-project.eu/
http://robmosys.eu/ € 8,000,000.00
€ 7,504,236.25
• ABB AB Sweden
• Delft University of Technology
• FH Aachen University of Applied Sciences
• Fraunhofer-Institut für Produktionstechnik und Automatisierung IPA
• Fundación Tecnalia Research & Innovations
• IT University of Copenhagen
ROS Industrial
- 129. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Known Issue of ROS 1.0
• Realtime performance
• Node management
• Master (roscore)
• Multi-robots capability (Swarm)
• Small processors
• Communication Priority
• Communication Security
• Unreliable network connectivity
Source: http://blog.ascens-ist.eu/wp-content/uploads/2011/03/13marxbots.jpg
ROS 2.0
- 131. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Source: https://roscon.ros.org/2014/wp-content/uploads/2014/07/ROSCON-2014-Next-Generation-of-ROS-on-top-of-DDS.pdf
=
less time
spent here
more time to spent here
ROS 2.0
- 133. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
User Code
ROS Client Lib
rclpy rclcpp Third Party
ROS Middleware
OpenSplice
RMW Impl.
FastRTPS
RMW Impl.
Third Party
RMW Impl.
PrismTech
OpenSplice
eProsima
FastRTPS
Third Party
DDS
User
Applications
Client
Wrapper
Middleware
DDS
DDS
agnostic
ROS
agnostic
ROS 2.0
- 140. https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
Q & A
Website: https://hypharosworkshop.wordpress.com/
Github: https://github.com/Hypha-ROS/hypha-racecar
FB Page: https://www.facebook.com/HyphaROS/
Youku: http://i.youku.com/hypha
Gmail: hypha.ros@gmail.com
WeChat (ID): HyphaROS