If you think there's been a lot of talk about Augmented Reality and Virtual Reality this year, 2018 is going to blow you away. Apple with ARkit, Google with ARCore , Microsoft with HoloLens, Facebook with Oculus and many others are working to transform our Reality with new products and services in the not-too-distant future. Therefore Apple, Microsoft, Google and Facebook is approaching AR/VR from different perspectives and in this session we will try to understand how these different technologies work and which best suits the different areas (industry 4.0, tourism, healthcare, ...)
Matteo Valoriani - How Augment your Reality: different perspective on the Reality-Virtuality Continuum - Codemotion Milan 2017
1. How Augment your Reality:
Different perspective on the Reality / Virtuality Continuum
Matteo Valoriani
CODEMOTION MILAN - SPECIAL EDITION
10 – 11 NOVEMBER 2017
2. How Augment your Reality:
Different perspective on the Reality / Virtuality Continuum
3. Nice to Meet You
matteo.valoriani@fifthingenium.com
Matteo Valoriani, PhD
CEO of FifthIngenium
mvaloriani at gmail.com
@MatteoValoriani
Slideshare: www.slideshare.net/MatteoValoriani
Linkedin: https://it.linkedin.com/in/matteovaloriani
Blog: http://fifthingenium.com/blog
GitHub: https://github.com/mvaloriani
4.
5. How we are arrived here
Virtual Continuum
VR
• Outside-in vs Inside-out
AR
• Visual Inertial Odometry (VIO)
• ARKit/ARCore
• HoloLens
Agenda
29. Pros
• Accuracy (you can add
more trackers around
the room to increase
accuracy)
• Latency very low (reduce
VR sickness).
Cons
• Setup
• Occlusion
• Limitation of the
cameras (field of view)
31. • .
Pros
• Simplicity / No Setup
• Less hardware needed (no
external camera)
• No limit of space
(potentially)
Cons
• Reduced accuracy and
latency
• Only the object in front to
the user can be tracked
• Powerful hardware
39. Visual Inertial Odometry (VIO)
6dof in real time :
• 3D of xyz motion (translation)
• 3D of pitch/yaw/roll (rotation)
Pose is tracked:
• via the Camera system, by matching a point in the real world to a pixel on the camera sensor each frame.
• by the Inertial system (accelerometer & gyroscope — Inertial Measurement Unit or IMU)
The output of both of those systems are then combined via a Kalman Filter which determines which of the two
systems is providing the best estimate of your “real” position.
42. The secret here is to have *really* good IMU error removal: Dead-reckoning
43.
44. • .
TrueDepth Camera
Face tracking.
Scene Understanding
Detection of horizontal planes (tables, floors,
…) and tracking of object.
Lighting Estimation
Estimation of light available in a scene in
order to apply the correct amount of lighting
to virtual objects.
Hardware and Rendering
Optimizations
Apple A9, A10, and A11 processors.
45.
46.
47. • .
Motion tracking
Estimates a phone’s relative location based
on internal sensors and video footage
Scene Understanding
Uses the camera to detect flat surfaces
Lighting Estimation
Helps virtual props cast accurate shadows
and otherwise fit in with their surroundings
Anchoring objects
48.
49.
50. • .
ARCore vs Tango
Tango’s dedicated infrared depth-
sensing camera to create detailed 3D
meshes.
ARCore has to estimate scale based
on the camera feed, while Tango
directly measures distance.
Nikhil Chandhok (AR director of
product): “ARCore is for all the apps
that we think that users want”
57. Holographic Processing Unit v 1.0
TSMC-fabricated 28 nm co-processor.
24 Tensilica DSP cores (12 clusters)
65 million logic gates (used 50%)
8 MB of SRAM
1GB DRAM
1 Trillion Operation per second
Sensor aggregator with gesture and environment
processing
200x over software implementation
Low Power (<10 Watts)
12 mm
12mm
60. • .
I've seen things you
people wouldn't believe.
Attack ships on fire off the
shoulder of Orion.
I watched C-beams glitter in the
dark near the Tannhäuser Gate.
All those moments will be lost in
time, like tears in rain. Time to
die.