SlideShare uma empresa Scribd logo
1 de 138
Baixar para ler offline
Lecture 2: AR T h l
L       2     Technology

         Mark Billinghurst
   mark.billinghurst@hitlabnz.org

             July 2011

  COSC 426: Augmented Reality
Key Points from Lecture 1
Augmented Reality Definition
Defining Characteristics [Azuma 97]
  Combines Real and Virtual Images
                               g
   - Both can be seen at the same time
  Interactive in real-time
   - Virtual content can be interacted with
  Registered in 3D
    g
   - Virtual objects appear fixed in space
What is not Augmented Reality?
Location-based services
Barcode detection (QR-codes)
B     d d        i (QR d )
Augmenting still images
  g         g        g
Special effects in movies
…
… but they can be combined with AR!
Milgram’s Reality-Virtuality Continuum

                       Mixed Reality


   Real        Augmented           Augmented          Virtual
Environment    Reality (AR)       Virtuality (AV)   Environment




              Reality - Virtuality (RV) Continuum
Metaverse
AR History Summary
1960’s – 80’s: Early Experimentation
1980 s 90 s:
1980’s – 90’s: Basic Research
  Tracking, displays
1995 – 2005: Tools/Applications
  Interaction, usability, theory
                       y       y
2005 - : Commercial Applications
  Games, M di l Industry
  G      Medical, I d
Applications
Medicine
Manufacturing
Information overlay
Architecture
Museum
Marketing
Gaming
Interaction Design Process
Interaction Design is All About You

   Users should be
   involved throughout
   the Design Process
   Co s de all the eeds
   Consider a t e needs
   of the user
Building Compelling AR Experiences
B ildi   C    lli      E    i

          experiences
                         Usability

          applications   Interaction


             tools       Authoring


          components     Tracking, Display
AR Technology
Building Compelling AR Experiences

            experiences

            applications

               tools

            components     Display, Tracking



                                      Sony CSL © 2004
AR Technology
Key Technologies
  Display
     p y
                  Tracking            Display
  Tracking
  Input
  Processing                 Processing


                  Input
AR Displays
AR Displays

                                                                    AR
                                                              Visual Displays


                                           Primarily Indoor                                              Primarily Outdoor
                                            Environments                                              (Daylight) Environments


                  Not Head-Mounted                             Head-Mounted               Head-Mounted                   Not Head Mounted
                                                               Display (HMD)              Display (HMD)                (e.g. vehicle mounted)


 Virtual Images               Projection CRT Display           Liquid Crystal        Cathode Ray Tube (CRT)               Projection Display
                                                                                  or Virtual Retinal Display (VRD)
                                                                                                        p y(     )    Navigational Aids in Cars
                                                                                                                            g
seen off windows                using beamsplitter             Displays LCDs          Many Military Applications     Military Airborne Applications
                                                                                      & Assistive Technologies


  e.g. window                        e.g. Reach-In            e.g. Shared Space             e.g. WLVA                       e.g. Head-Up
   reflections                                                   Magic Book                 and IVRD                        Display (HUD)
Head Mounted Displays
Head Mounted Displays (HMD)
   -   Display and Optics mounted on Head
   -   May or may not fully occlude real world
   -   Provide full-color images
   -   Considerations
        •   Cumbersome to wear
        •   Brightness
        •   Low power consumption
        •   Resolution limited
        •   Cost is high?
                      g
Types of Head Mounted Displays

       Occluded
                      See-thru




        Multiplexed
Immersive VR Architecture
                                                                          Virtual
                                                                          World
             head position/orientation
                                             Head       Non see-thru
                                            Tracker    Image source
                                                          & optics




          Host      Data Base   Rendering
                                              Frame
        Processor
        P            Model
                     M d l       Engine
                                 E i
                                              Buffer

                                                                virtual
to network                                                      object
                                            Display
                                            Driver
See-thru AR Architecture

        head position/orientation
                                          Head           see-thru
                                         Tracker        combiner
                                                                    real world




      Host       Data Base   Rendering
                                           Frame
    Processor
    P             Model
                  M d l       Engine
                              E i
                                           Buffer


to network                                                          Virtual Image
                                         Display                    superimposed
                                         Driver                     over real world
                                                                    object
                                                    Image source
                                                       g
Optical see-through head-mounted display
          Virtual images
          from monitors


  Real
  World
             Optical
             Combiners
Optical See-Through HMD
Optical see-through HMDs
                 Virtual Vision VCAP




Sony Glasstron
DigiLens
                     Compact HOE
                     C
                         Solid state optics
                         Switchable Bragg Grating
                         Stacked SBG
                         Fast switching
                         Ultra compact




www.digilens.com
The Virtual Retinal Display




Image scanned onto retina
  age sca e o to et a
Commercialized through Microvision
  Nomad System - www.mvis.com
Strengths of optical AR
Simpler (cheaper)
Direct view of real world
Di      i    f    l    ld
  Full resolution, no time delay (for real world)
  Safety
  Lower distortion
No eye displacement
  but COASTAR video see-through avoids this
Video AR Architecture
                                                        Head-mounted
                                                       camera aligned to
             head position/orientation                   display optics
                                                                             Video image
                                            Head
                                           Tracker                           of real world

                                 Video
                               Processor


         Host       Graphics    Digital
                                             Frame
       Processor
       P            renderer
                       d        Mixer
                                Mi
                                             Buffer



to network
                                           Display
                                           Driver
                                                                           Virtual image
                                                                           inset into
                                                       Non see-thru
                                                                           video of real
                                                      Image source
                                                                           world
                                                         & optics
Video see-through HMD
     Video
     cameras         Video
                         Graphics

Monitors              Combiner
Video See-Through HMD
Video see-through HMD




MR Laboratory’s COAS A HMD
                ’ COASTAR
(Co-Optical Axis See-Through Augmented Reality)
Parallax-free
Parallax free video see through HMD
                    see-through
TriVisio
www.trivisio.com
               p
Stereo video input
  PAL resolution cameras
2 x SVGA displays
  30 degree FOV
  User adjustable convergence
$6,000 USD
Vuzix Display

www.vuzix.com
Wrap 920
$350 USD
Twin 640 x 480 LCD displays
31 degree diagonal field of view
Weighs less than three ounces
Strengths of Video AR
True occlusion
  Kiyokawa optical display that supports occlusion
    y       p         p y         pp
Digitized image of real world
  Flexibility
  Fl b l in composition
  Matchable time delays
  More registration, calibration strategies
Wide FOV is easier to support
Optical vs. Video AR Summary
Both have proponents
Video is more popular today?
  Likely because lack of available optical products
       y                            p      p
Depends on application?
  Manufacturing: optical i cheaper
  M f         i      i l is h
  Medical: video for calibration strategies
Eye multiplexed AR Architecture

        head position/orientation
                                         Head
                                        Tracker                   real world




      Host      Data Base   Rendering
                                          Frame
    Processor
    P            Model
                 M d l       Engine
                             E i
                                          Buffer


to network
                                        Display                     Virtual Image
                                        Driver                      inset into
                                                                    real world scene
                                                   Opaque
                                                   Image source
Virtual Image ‘inset’ into real
Virtual Vision Personal Eyewear
Virtual image inset into real world
Spatial/Projected AR
Spatial Augmented Reality




Project onto irregular surfaces
  Geometric Registration
  Projector blending, High dynamic range
Book: Bimber, Rasker “Spatial Augmented Reality”
                       p        g             y
Projector-based AR
                               User (possibly
                               head-tracked)




                                  Projector

                       Examples:
Real objects
                       Raskar,
                       Raskar MIT Media Lab
with retroreflective
                       Inami, Tachi Lab, U. Tokyo
covering
Example of projector-based AR




       Ramesh Raskar, UNC, MERL
                    ,    ,
Example of projector-based AR




    Ramesh Raskar, UNC Chapel Hill
The I/O Bulb




Projector + C
P j         Camera
  John Underkoffler, Hiroshi Ishii
  MIT Media Lab
Head Mounted Projector




Head Mounted Projector
  J
  Jannick Rolland (
                  (UCF)
                      )
Retro-reflective Material
  Potentially portable
Head Mounted Projector




NVIS P 50 HMPD
     P-50
  1280x1024/eye
  Stereoscopic
  Stere sc ic
  50 degree FOV
  www.nvis.com
          i
HMD vs. HMPD




Head Mounted Display   Head Mounted Projected Display
Pico Projectors




Microvision - www.mvis.com
3M, Samsung, Phili etc
3M S          Philips, t
MIT Sixth Sense




Body worn camera and projector
   p       p          y     p j
http://www.pranavmistry.com/projects/sixthsense/
Other AR Displays
Video Monitor AR
       Video                  Stereo
       cameras      Monitor   g
                              glasses




            Video

Graphics         Combiner
Virtual Showcase
Mirrors on a projection table
  Head
  H d tracked stereo
           k d
  Up to 4 users
  Merges graphic and real objects
  M           hi   d    l bj
  Exhibit/museum applications
Fraunhofer Institute (2001)
  Bimber, Frohlich
Augmented Paleontology




 Bimber et. al. IEEE Computer Sept. 2002
Alternate Displays




LCD Panel      Laptop        PDA
Handheld Displays
Mobile Phones
  Camera
  Display
  Input
Other Types of AR Display
Audio
  spatial sound
   p
  ambient audio
Tactile
T til
  physical sensation
Haptic
  virtual touch
Haptic Input




AR Haptic Workbench
  CSIRO 2003 – Adcock et al
                      et. al.
Phantom




Sensable Technologies (www.sensable.com)
6 DOF Force Feedback Device
AR Haptic Interface




Phantom, ARToolKit, Magellan
                      g
AR Tracking and Registration
Tracking
  Locating the users viewpoint
         g               p
  Position (x,y,z)
  Orientation (r p y)
               (r,p,y)
Registration
  Positioning virtual object wrt real world
Tracking Requirements




Head Stabilized     Body Stabilized      World Stabilized
  Augmented Reality Information Display
     World Stabilized
     Body Stabilized            Increasing Tracking
                                Requirements
     Head Stabilized
Tracking Technologies
•   Mechanical
•            g
    Electromagnetic
•   Optical
•   Acoustic
•   Inertial d dead
    I ti l and d d reckoning
                      k i
•   GPS
•   Hybrid
AR Tracking Taxonomy
                                           AR
                                        TRACKING

                           Indoor                                  Outdoor
                         Environment                             Environment


         Limited Range             Extended Range       Low Accuracy &   High Accuracy
                                                          Not Robust       & Robust


Low Accuracy     High Accuracy         Many Fiducials   Not Hybridized   Hybrid Tracking
 at 15-60 Hz     & High Speed           in space/time       GPS or          GPS and
                     Hybrid
                     H b id                  but
                                             b t          Camera or
                                                          C                Camera and
                                                                           C         d
                    Tracking               no GPS         Compass           Compass


e.g.
e g AR Toolkit     e.g.
                   e g IVRD              e.g.
                                         e g HiBall       e.g.
                                                          e g WLVA         e.g.
                                                                           e g BARS
Tracking Types

Magnetic   Inertial        Ultrasonic       Optical      Mechanical
Tracker
T k        Tracker
           T k              Tracker
                            T k             Tracker
                                            T k           Tracker


                        Specialized      Marker-Based
                                         Marker Based     Markerless
                         Tracking          Tracking        Tracking



                      Edge-Based      Template-Based    Interest Point
                              g
                       Tracking                 g
                                         Tracking                 g
                                                           Tracking
Tracking Systems
Mechanical Tracker
Magnetic Tracker
Ultrasonic Tracker
Inertial Tracker
Vision (Optical Tracking)
  Specialized (Infrared, Retro-Reflective)
  Monocular (DVCam, Webcam)
  M        l (DVC        W b     )
Mechanical Tracker
Idea:
Id mechanical arms with joint sensors
      h     l         h




                                Microscribe


++: high accuracy haptic feedback
         accuracy,
-- : cumbersome, expensive
Magnetic Tracker
Idea: difference between a magnetic transmitter
and a receiver




    Flock of Birds (Ascension)



++: 6DOF robust
     6DOF, b
-- : wired, sensible to metal, noisy, expensive
                                   y p
Inertial Tracker
                     I      lT k
    Idea: measuring linear and angular orientation rates
    (accelerometer/gyroscope)




IS300 (Intersense)
                                  Wii Remote

    ++: no transmitter, cheap small high frequency wireless
              transmitter cheap, small,  frequency,
    -- : drift, hysteris only 3DOF
Ultrasonics Tracker
  Idea: Time of Flight or Phase Coherence Sound Waves
                          Phase-Coherence




Ultrasonic
Logitech                                           IS600

  ++: Small, Cheap
  -- : 3DOF, Line of Sight, Low resolution, Affected
  Environment Conditon (pressure, temperature)
Global Positioning System (GPS)
Created by US in 1978
  Currently 29 satellites
Satellites send position + time
GPS Receiver positioning
               p         g
  4 satellites need to be visible
  Differential time of arrival
  Triangulation
Accuracy
       y
  5-30m+, blocked by weather, buildings etc
Problems with GPS
Takes time to get satellite fix
   Satellites moving around
Earths atmosphere affects signal
            p               g
   Assumes consistent speed (the speed of light).
   Delay depends where you are on Earth
   Weather effects
Signal reflection
   Multi-path reflection off buildings
Signal blocking
   Trees, buildings, mountains
   T      b ildi         t i
Satellites send out bad data
   Misreport their own position
Accurate to < 5cm close to base station (22m/100 km)
Expensive - $20-40,000 USD
Optical Tracking
Optical Tracker
Idea: Image Processing and Computer Vision
         g           g        p
Specialized
      Infrared, Retro-Reflective, Stereoscopic
        f               f         S




ART                                              Hi-Ball




Monocular Based Vision Tracking
                              g
Outside-In vs. Inside-Out Tracking
Optical Tracking Technologies

Scalable active trackers
  InterSense IS-900, 3rd Tech HiBall
                                         3rd Tech, I
                                         3 d T h Inc.
Passive optical computer vision
  Line of sight, may require landmarks
          sight
  Can be brittle.
  Computer vision i computationally-intensive
  C             i i is          i ll i    i
HiBall Tracking System (3rd Tech)
Inside-Out Tracker
       O
  $50K USD
Scalable over large area
  Fast d t (2000H )
  F t update (2000Hz)
  Latency Less than 1 ms.
Accurate
  Position 0.4mm RMS
           0 4mm
  Orientation 0.02° RMS
COSC 426 Lect 2. - AR Technology
Starting simple: Marker tracking
Has been done for more than 10 years
Several open source solutions exist
S       l              l i      i
Fairly simple to implement
     y    p        p
  Standard computer vision methods
A rectangular marker provides 4 corner points
          l      k       id              i
  Enough for pose estimation!
Marker Based Tracking: ARToolKit




http://artoolkit.sourceforge.net/
Coordinate Systems
C d        S
Marker T k
M k Tracking – O
               Overview
Marker Tracking – Fiducial Detection
Threshold the whole image to black and white
Search scanline by scanline for edges (white to black)
Follow edge until either
  Back to starting pixel
  Image border
Check for size
  Reject fiducials early that are too small (or too large)
Marker Tracking – Rectangle Fitting
Start with an arbitrary point “x” on the contour
S      ih       bi         i “ ”      h
The point with maximum distance must be a corner c0
Create a diagonal through the center
C         d       l h     h h
Find points c1 & c2 with maximum distance left and right of diag.
New diagonal from c1 to c2
Find point c3 right of diagonal with maximum distance
Marker Tracking – Pattern checking
Calculate homography using the 4 corner points
   “Direct Linear Transform” algorithm
   Maps normalized coordinates to marker coordinates
      p
   (simple perspective projection, no camera model)
Extract pattern by sampling
Check pattern
   Id (implicit encoding)
   Template (
   T     l    (normalized cross correlation)
                     li d            l i )
Marker Tracking – Corner refinement
 Refine corner coordinates
   Critical for high quality tracking
   Remember: 4 points is the bare minimum!
   So these 4 points should better be accurate…
 Detect sub-pixel coordinates
   E.g. Harris corner detector
     g
   - Specialized methods can be faster and more accurate
   Strongly reduces jitter!
         gy          j
 Undistort corner coordinates
   Remove radial distortion from lens
   R        di l di t ti f       l
Marker
M k tracking – P
        k      Pose estimation
Calculates marker position and rotation
C
relative to the camera
Initial estimation directly from homography
  Very fast, but coarse
        fast
  Jitters a lot…
Refinement via G
R fi        i Gauss-Newton iteration
                    N      i     i
  6 parameters (3 for position, 3 for rotation) to refine
  At each iteration we optimize on the reprojection error
Coordinates for Marker Tracking
Coordinates for Marker Tracking
•Camera Observed Screen
   Marker Ideal Screen Screen
•Ideal ScreenCamera
•Marker          Observed
•Perspective model (barrel shape)
   •Goal
•Correspondence of
•Nonlinear function 4 vertices
•Obtained fromTranslation
   •Rotation & Camera Calibration
•Obtained fromT processing
    R t ti
•Real time image    l ti
                Camera Calibration
From Marker To Camera
    F    M k T C
Rotation & Translation




  TCM : 4x4 transformation matrix
       from marker coord. to camera coord.
Tracking challenges in ARToolKit



  Occlusion            Unfocused camera, Dark/unevenly lit         Jittering
(image by M. Fiala)        motion blur   scene, vignetting   (Photoshop illustration)




                                                              Image noise
False positives and inter-marker confusion              (e.g. poor lens, block coding /
                                                          compression, neon tube)
                      (image by M. Fiala)
Tracking, Tracking, Tracking
Other Marker Tracking Libraries
arTag
  T
  http://www.artag.net/
ARToolKitPlus [Discontinued]
  http://studierstube.icg.tu-
  graz.ac.at/handheld_ar/artoolkitplus.php
  graz ac at/handheld ar/artoolkitplus php
stbTracker
  http://studierstube.icg.tu-
  htt // t di t b i t
  graz.ac.at/handheld_ar/stbtracker.php
MXRToolKit
  http://sourceforge.net/projects/mxrtoolkit/
Markerless Tracking
COSC 426 Lect 2. - AR Technology
Markerless Tracking
   No more Markers!          Markerless Tracking
Magnetic T k
M     i Tracker   Inertial
                  I    i l        Ultrasonic
                                  Ul      i          Optical
                                                     O i l
                  Tracker          Tracker           Tracker


                               Specialized       Marker-Based      Markerless
                                Tracking          Tracking         Tracking



                             Edge-Based        Template-Based   Interest Point
                              Tracking            Tracking        Tracking
Natural feature tracking
Tracking from features of the surrounding
environment
  Corners, edges, blobs, ...
Generally more diffi l than marker tracking
G     ll       difficult h     k       ki
  Markers are designed for their purpose
                   g             p p
  The natural environment is not…
Less well established methods
     well-established
Usually much slower than marker tracking
Natural Feature Tracking
                                     Features Points
 Use Natural Cues of Real Elements
                                             Contours
    Edges
    Surface Texture
    Interest Points
 Model or Model-Free
 ++: no visual pollution


                                              Surfaces
Texture Tracking
Tracking by d
         T k b detection
This is what most trackers do
                           do…       Camera Image
                                               g



Targets are detected every frame
                                   Keypoint detection
                                     yp
Popular because
tracking and detection             Descriptor creation

are solved simultaneously
                                     and matching
                                       d     t hi




                                    Outlier Removal



                                    Pose estimation
                                    and refinement




                                         Pose
Natural feature tracking – What is a keypoint?
  It depends on the detector you use!
  For high performance use the FAST corner
  detector
    Apply
    A l FAST t all pixels of your i
                to ll i l f         image
    Obtain a set of keypoints for your image
     - R d
       Reduce the amount of corners using non-maximum suppression
               h          f
    Describe the keypoints

     E. Rosten and T. Drummond (May 2006). "Machine learning for high‐speed corner detection". 
Corner keypoint
Natural feature tracking – Descriptors
 Again depends on your choice of a descriptor!
 Can use SIFT
   Estimate the d i
   E i       h dominant keypoint
                           k    i
   orientation using gradients
   Compensate for
   C m ensate f r detected
   orientation
   Describe the keypoints in terms
   of the gradients surrounding it


                        Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D., 
              Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones.
               IEEE Transactions on Visualization and Computer Graphics, May/June, 2010 
NFT – D b
           Database creation
Offline step
           p
Searching for corners in a static image
For robustness look at corners on multiple scales
  Some corners are more descriptive at larger or smaller scales
  We d ’t k
  W don’t know how far users will be from our image
                 h   f          ill b f         i
Build a database file with all descriptors and their
position on the original i
    ii       h      i i l image
NFT – R l
             Real-time tracking
                           k
Search for keypoints                       Camera Image


in the video image
Create the d
C t th descriptorsi t                    Keypoint detection



Match the descriptors from the           Descriptor creation
                                               p
live video against those                   and matching


in the database
                                          Outlier Removal
                                          O tli R       l
  Brute force is not an option
  Need the speed-up of special            Pose estimation

  data structures                         and refinement


   - E.g., we use multiple spill trees
                                               Pose
NFT – O l removal
             Outlier   l
Cascade of removal techniques
Start with cheapest, finish with most
expensive…
  First simple geometric tests
  - E.g., line tests
     • Select 2 points to form a line
     • Check all other points being on correct side of line
  Then, homography-based tests
NFT – P
            Pose refinement
                   f
Pose from homography makes good starting point
Based on Gauss-Newton iteration
  Try to minimize the re-projection error of the keypoints
Part of tracking pipeline that mostly benefits
from floating point usage
Can still be implemented effectively in fixed point
Typically 2-4 iterations are enough…
NFT – R l
           Real-time tracking
                         k
Search for keypoints               Camera Image

in the video image
                   p
Create the descriptors           Keypoint detection


Match the descriptors from the
live video against those         Descriptor creation
                                   and matching

in the database
Remove the keypoints that         Outlier Removal


are outliers                      Pose estimation

Use h
U the remaining k
               i i keypoints
                         i
                                  and refinement



to calculate the pose
                                       Pose
of the camera
  f h
NFT – R l
        Results




          Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D., 
Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones.
 IEEE Transactions on Visualization and Computer Graphics, May/June, 2010 
Edge Based Tracking
RAPiD [Drummond et al. 02]
  Initialization, Control Points, Pose Prediction (Global Method)
Line Based Tracking
 Visual Servoing [Comport et al. 2004]
Model Based Tracking
OpenTL - www.opentl.org
  General purpose library for model based visual tracking
OpenTL Features
COSC 426 Lect 2. - AR Technology
Visual Modalities Used For Tracking
The Tracking Pipeline
Marker vs. natural feature tracking
 Marker tracking
   Usually requires no database to be stored
   Markers can be an eye-catcher
   Tracking is less demanding
           g                g
   The environment must be instrumented with markers
   Markers usually work only when fully in view
                  y         y          y
 Natural feature tracking
   A database of keypoints must be stored/downloaded
   Natural feature targets might catch the attention less
   Natural f t
   N t l feature targets are potentially everywhere
                   t    t        t ti ll            h
   Natural feature targets work also if partially in view
Hybrid Tracking
Example: Outdoor Hybrid Tracking
 Combines
   computer vision
     - natural feature tracking
   inertial gyroscope sensors
 Both correct for each other
   Inertial gyro - provides frame to frame
   prediction of camera orientation
   Computer vision - correct for gyro drift
Outdoor AR Tracking System




You, Neumann, Azuma outdoor AR system (1999)
Robust Outdoor Tracking




Hybrid T ki
H b id Tracking
  Computer Vision, GPS, inertial
Going Out
  Reitmayer & Drummond (Univ. Cambridge)
Handheld Display
Registration
The Registration Problem
Virtual and Real must stay properly aligned
If not:
     t
  Breaks the illusion that the two coexist
  Prevents acceptance of many serious applications
Sources of registration errors
Static errors
S
  Optical distortions
  Mechanical misalignments
  Tracker errors
  Incorrect viewing parameters
Dynamic errors
  System delays (largest source of error)
   - 1 ms d l = 1/3 mm registration error
          delay           i t ti
Reducing static errors
Distortion compensation
Manual adjustments
View-based or direct measurements
  [Azuma94] [Caudell92] [Janin93] etc.
Camera calibration (video)
  [ARGOS94] [Bajura93] [Tuceryan95] etc.
View Based Calibration (Azuma 94)
Dynamic errors
                             Application Loop

               x,y,z
Tracking       r,p,y
                          Calculate             Render       Draw to
                          Viewpoint             Scene        Display
                          Simulation




20 Hz = 50ms             500 Hz = 2ms       30 Hz = 33ms   60 Hz = 17ms

                Total Delay = 50 + 2 + 33 + 17 = 102 ms
                       1 ms delay = 1/3 mm = 33mm error
Reducing dynamic errors (1)

Reduce system lag
  [Olano95] [Wloka95a] [Regan SIGGRAPH99]
Reduce apparent lag
  Image deflection [Burbidge89] [Regan94] [So92]
  [Kijima
  [Kiji ISMR 2001]
  Image warping [Mark 3DI 97]
Reducing System Lag
                        Application Loop

             x,y,z
Tracking     r,p,y
                      Calculate        Render       Draw to
                      Viewpoint        Scene        Display
                      Simulation




Faster Tracker       Faster CPU      Faster GPU   Faster Display
Reducing Apparent Lag
   Virtual Display                        Virtual Display
                              x,y,z
                                y
     Physical
     Ph i l                   r,p,y
                                            Physical
      Display                                Display
    (640x480)                              (640x480)
                           Tracking
    1280 x 960              Update         1280 x 960

Last known position                   Latest position
                       Application Loop

             x,y,z
Tracking     r,p,y
                     Calculate        Render            Draw to
                     Viewpoint
                         p            Scene             Display
                                                           p y
                     Simulation
Reducing dynamic errors (2)
Match input streams (video)
  Delay video of real world to match system lag
Predictive Tracking
  [Azuma94] [Emura94]
  Inertial sensors helpful




                               Azuma / Bishop 1994
                                 u a     s op 99
Predictive Tracking
Position
                                 Now




                     Past              Future
                                                Time

      Can predict up to 80 ms in future (Holloway)
Predictive Tracking (Azuma 94)
Wrap-up
Tracking and Registration are key problems
Registration error
  Measures against static error
  Measures against dynamic error
  M           i    d      i
AR typically requires multiple tracking technologies
    yp     y q             p          g         g
Research Areas: Hybrid Markerless Techniques,
Deformable Surface, Mobile, Outdoors
             Surface Mobile
More Information
• M k Billi h t
  Mark Billinghurst
  – mark.billinghurst@hitlabnz.org
• Websites
  – www.hitlabnz.org
        hi l b

Mais conteúdo relacionado

Mais procurados

Camera-based sensory substitution and augmented reality for the blind (ACIVS ...
Camera-based sensory substitution and augmented reality for the blind (ACIVS ...Camera-based sensory substitution and augmented reality for the blind (ACIVS ...
Camera-based sensory substitution and augmented reality for the blind (ACIVS ...Peter Meijer
 
Agile2011-Agile Education by Object Game
Agile2011-Agile Education by Object Game Agile2011-Agile Education by Object Game
Agile2011-Agile Education by Object Game Tsuyoshi Ushio
 
Augmented Reality: Beyond the Hype
Augmented Reality: Beyond the HypeAugmented Reality: Beyond the Hype
Augmented Reality: Beyond the HypePaul Coulton
 
セミナー資料(20OCT2011)Power Point
セミナー資料(20OCT2011)Power Pointセミナー資料(20OCT2011)Power Point
セミナー資料(20OCT2011)Power PointTsuyoshi Horigome
 
Object modeling in robotic perception
Object modeling in robotic perceptionObject modeling in robotic perception
Object modeling in robotic perceptionMoniqueO Opris
 
Providing new customer_experience(final)
Providing new customer_experience(final)Providing new customer_experience(final)
Providing new customer_experience(final)Tsukasa Makino
 
Good code-isnt-enough
Good code-isnt-enoughGood code-isnt-enough
Good code-isnt-enoughSkills Matter
 
Future of Mobile Augmented Reality (Zenitum's View Point)
Future of Mobile Augmented Reality (Zenitum's View Point)Future of Mobile Augmented Reality (Zenitum's View Point)
Future of Mobile Augmented Reality (Zenitum's View Point)DoubleMe, Inc.
 
Wired Ovita E Tech
Wired Ovita E TechWired Ovita E Tech
Wired Ovita E TechWIRED staff
 
Computer generated holography as a generic display technology
Computer generated holography as a generic display technologyComputer generated holography as a generic display technology
Computer generated holography as a generic display technologyPritam Bhansali
 

Mais procurados (15)

Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 WebRaskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
 
Scd 2080 r
Scd 2080 rScd 2080 r
Scd 2080 r
 
Camera-based sensory substitution and augmented reality for the blind (ACIVS ...
Camera-based sensory substitution and augmented reality for the blind (ACIVS ...Camera-based sensory substitution and augmented reality for the blind (ACIVS ...
Camera-based sensory substitution and augmented reality for the blind (ACIVS ...
 
Alternativa3D
Alternativa3DAlternativa3D
Alternativa3D
 
Sccg Many Projects Layout03
Sccg Many Projects Layout03Sccg Many Projects Layout03
Sccg Many Projects Layout03
 
Alternativa3D_en
Alternativa3D_enAlternativa3D_en
Alternativa3D_en
 
Agile2011-Agile Education by Object Game
Agile2011-Agile Education by Object Game Agile2011-Agile Education by Object Game
Agile2011-Agile Education by Object Game
 
Augmented Reality: Beyond the Hype
Augmented Reality: Beyond the HypeAugmented Reality: Beyond the Hype
Augmented Reality: Beyond the Hype
 
セミナー資料(20OCT2011)Power Point
セミナー資料(20OCT2011)Power Pointセミナー資料(20OCT2011)Power Point
セミナー資料(20OCT2011)Power Point
 
Object modeling in robotic perception
Object modeling in robotic perceptionObject modeling in robotic perception
Object modeling in robotic perception
 
Providing new customer_experience(final)
Providing new customer_experience(final)Providing new customer_experience(final)
Providing new customer_experience(final)
 
Good code-isnt-enough
Good code-isnt-enoughGood code-isnt-enough
Good code-isnt-enough
 
Future of Mobile Augmented Reality (Zenitum's View Point)
Future of Mobile Augmented Reality (Zenitum's View Point)Future of Mobile Augmented Reality (Zenitum's View Point)
Future of Mobile Augmented Reality (Zenitum's View Point)
 
Wired Ovita E Tech
Wired Ovita E TechWired Ovita E Tech
Wired Ovita E Tech
 
Computer generated holography as a generic display technology
Computer generated holography as a generic display technologyComputer generated holography as a generic display technology
Computer generated holography as a generic display technology
 

Semelhante a COSC 426 Lect 2. - AR Technology

2013 426 Lecture 2: Augmented Reality Technology
2013 426 Lecture 2:  Augmented Reality Technology2013 426 Lecture 2:  Augmented Reality Technology
2013 426 Lecture 2: Augmented Reality TechnologyMark Billinghurst
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented RealityKaviyaraj R
 
2014 COSC 426 Lecture 2: Augmented Reality Technology
2014 COSC 426 Lecture 2: Augmented Reality Technology2014 COSC 426 Lecture 2: Augmented Reality Technology
2014 COSC 426 Lecture 2: Augmented Reality TechnologyMark Billinghurst
 
COMP 4010 - Lecture 8 AR Technology
COMP 4010 - Lecture 8 AR TechnologyCOMP 4010 - Lecture 8 AR Technology
COMP 4010 - Lecture 8 AR TechnologyMark Billinghurst
 
Augmented reality
Augmented realityAugmented reality
Augmented realitykaviyaraj
 
Simulating Presence: BIM to Virtual Reality
Simulating Presence: BIM to Virtual RealitySimulating Presence: BIM to Virtual Reality
Simulating Presence: BIM to Virtual RealityJason Halaby
 
426 lecture6a osgART Development
426 lecture6a osgART Development426 lecture6a osgART Development
426 lecture6a osgART DevelopmentMark Billinghurst
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented RealityAjay Sankar
 
Droidcon2013 miracast final2
Droidcon2013 miracast final2Droidcon2013 miracast final2
Droidcon2013 miracast final2Droidcon Berlin
 
Tomer kimhi mobmodcon-nov2015-integrating new camera hardware
Tomer kimhi mobmodcon-nov2015-integrating new camera hardwareTomer kimhi mobmodcon-nov2015-integrating new camera hardware
Tomer kimhi mobmodcon-nov2015-integrating new camera hardwareRon Munitz
 
Aurea Vision & Aurea Consulting: technology proposal
Aurea Vision & Aurea Consulting: technology proposalAurea Vision & Aurea Consulting: technology proposal
Aurea Vision & Aurea Consulting: technology proposalAnna Doroshenko
 
COSC 426 Lect. 5 - Mobile AR
COSC 426 Lect. 5 - Mobile ARCOSC 426 Lect. 5 - Mobile AR
COSC 426 Lect. 5 - Mobile ARMark Billinghurst
 
Augmented Reality for Architects
Augmented Reality for ArchitectsAugmented Reality for Architects
Augmented Reality for ArchitectsGaurav Kudalkar
 

Semelhante a COSC 426 Lect 2. - AR Technology (20)

426 lecture2: AR Technology
426 lecture2: AR Technology426 lecture2: AR Technology
426 lecture2: AR Technology
 
2013 426 Lecture 2: Augmented Reality Technology
2013 426 Lecture 2:  Augmented Reality Technology2013 426 Lecture 2:  Augmented Reality Technology
2013 426 Lecture 2: Augmented Reality Technology
 
Augmented reality
Augmented realityAugmented reality
Augmented reality
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
2014 COSC 426 Lecture 2: Augmented Reality Technology
2014 COSC 426 Lecture 2: Augmented Reality Technology2014 COSC 426 Lecture 2: Augmented Reality Technology
2014 COSC 426 Lecture 2: Augmented Reality Technology
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
COMP 4010 - Lecture 8 AR Technology
COMP 4010 - Lecture 8 AR TechnologyCOMP 4010 - Lecture 8 AR Technology
COMP 4010 - Lecture 8 AR Technology
 
AR(mobas2012)s
AR(mobas2012)sAR(mobas2012)s
AR(mobas2012)s
 
Augmented reality
Augmented realityAugmented reality
Augmented reality
 
Simulating Presence: BIM to Virtual Reality
Simulating Presence: BIM to Virtual RealitySimulating Presence: BIM to Virtual Reality
Simulating Presence: BIM to Virtual Reality
 
426 lecture6a osgART Development
426 lecture6a osgART Development426 lecture6a osgART Development
426 lecture6a osgART Development
 
SVR2011 Keynote
SVR2011 KeynoteSVR2011 Keynote
SVR2011 Keynote
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
Droidcon2013 miracast final2
Droidcon2013 miracast final2Droidcon2013 miracast final2
Droidcon2013 miracast final2
 
Tomer kimhi mobmodcon-nov2015-integrating new camera hardware
Tomer kimhi mobmodcon-nov2015-integrating new camera hardwareTomer kimhi mobmodcon-nov2015-integrating new camera hardware
Tomer kimhi mobmodcon-nov2015-integrating new camera hardware
 
Aurea Vision & Aurea Consulting: technology proposal
Aurea Vision & Aurea Consulting: technology proposalAurea Vision & Aurea Consulting: technology proposal
Aurea Vision & Aurea Consulting: technology proposal
 
COSC 426 Lect. 5 - Mobile AR
COSC 426 Lect. 5 - Mobile ARCOSC 426 Lect. 5 - Mobile AR
COSC 426 Lect. 5 - Mobile AR
 
Augmented Reality for Architects
Augmented Reality for ArchitectsAugmented Reality for Architects
Augmented Reality for Architects
 
Nizar Romdhane, ARM
Nizar Romdhane, ARMNizar Romdhane, ARM
Nizar Romdhane, ARM
 

Mais de Mark Billinghurst

Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesMark Billinghurst
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the MetaverseMark Billinghurst
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseMark Billinghurst
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseMark Billinghurst
 
2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VR2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VRMark Billinghurst
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR SystemsMark Billinghurst
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR SystemsMark Billinghurst
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR PrototypingMark Billinghurst
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR InteractionMark Billinghurst
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR TechnologyMark Billinghurst
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: PerceptionMark Billinghurst
 
2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XRMark Billinghurst
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsMark Billinghurst
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseMark Billinghurst
 
Research Directions in Transitional Interfaces
Research Directions in Transitional InterfacesResearch Directions in Transitional Interfaces
Research Directions in Transitional InterfacesMark Billinghurst
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsMark Billinghurst
 
Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsMark Billinghurst
 

Mais de Mark Billinghurst (20)

Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR Experiences
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the Metaverse
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader Metaverse
 
2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VR2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VR
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems
 
ISS2022 Keynote
ISS2022 KeynoteISS2022 Keynote
ISS2022 Keynote
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR Systems
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception
 
2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive Analytics
 
Metaverse Learning
Metaverse LearningMetaverse Learning
Metaverse Learning
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole Metaverse
 
Research Directions in Transitional Interfaces
Research Directions in Transitional InterfacesResearch Directions in Transitional Interfaces
Research Directions in Transitional Interfaces
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research Directions
 
Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research Directions
 

Último

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 

Último (20)

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 

COSC 426 Lect 2. - AR Technology

  • 1. Lecture 2: AR T h l L 2 Technology Mark Billinghurst mark.billinghurst@hitlabnz.org July 2011 COSC 426: Augmented Reality
  • 2. Key Points from Lecture 1
  • 3. Augmented Reality Definition Defining Characteristics [Azuma 97] Combines Real and Virtual Images g - Both can be seen at the same time Interactive in real-time - Virtual content can be interacted with Registered in 3D g - Virtual objects appear fixed in space
  • 4. What is not Augmented Reality? Location-based services Barcode detection (QR-codes) B d d i (QR d ) Augmenting still images g g g Special effects in movies … … but they can be combined with AR!
  • 5. Milgram’s Reality-Virtuality Continuum Mixed Reality Real Augmented Augmented Virtual Environment Reality (AR) Virtuality (AV) Environment Reality - Virtuality (RV) Continuum
  • 7. AR History Summary 1960’s – 80’s: Early Experimentation 1980 s 90 s: 1980’s – 90’s: Basic Research Tracking, displays 1995 – 2005: Tools/Applications Interaction, usability, theory y y 2005 - : Commercial Applications Games, M di l Industry G Medical, I d
  • 10. Interaction Design is All About You Users should be involved throughout the Design Process Co s de all the eeds Consider a t e needs of the user
  • 11. Building Compelling AR Experiences B ildi C lli E i experiences Usability applications Interaction tools Authoring components Tracking, Display
  • 13. Building Compelling AR Experiences experiences applications tools components Display, Tracking Sony CSL © 2004
  • 14. AR Technology Key Technologies Display p y Tracking Display Tracking Input Processing Processing Input
  • 16. AR Displays AR Visual Displays Primarily Indoor Primarily Outdoor Environments (Daylight) Environments Not Head-Mounted Head-Mounted Head-Mounted Not Head Mounted Display (HMD) Display (HMD) (e.g. vehicle mounted) Virtual Images Projection CRT Display Liquid Crystal Cathode Ray Tube (CRT) Projection Display or Virtual Retinal Display (VRD) p y( ) Navigational Aids in Cars g seen off windows using beamsplitter Displays LCDs Many Military Applications Military Airborne Applications & Assistive Technologies e.g. window e.g. Reach-In e.g. Shared Space e.g. WLVA e.g. Head-Up reflections Magic Book and IVRD Display (HUD)
  • 18. Head Mounted Displays (HMD) - Display and Optics mounted on Head - May or may not fully occlude real world - Provide full-color images - Considerations • Cumbersome to wear • Brightness • Low power consumption • Resolution limited • Cost is high? g
  • 19. Types of Head Mounted Displays Occluded See-thru Multiplexed
  • 20. Immersive VR Architecture Virtual World head position/orientation Head Non see-thru Tracker Image source & optics Host Data Base Rendering Frame Processor P Model M d l Engine E i Buffer virtual to network object Display Driver
  • 21. See-thru AR Architecture head position/orientation Head see-thru Tracker combiner real world Host Data Base Rendering Frame Processor P Model M d l Engine E i Buffer to network Virtual Image Display superimposed Driver over real world object Image source g
  • 22. Optical see-through head-mounted display Virtual images from monitors Real World Optical Combiners
  • 24. Optical see-through HMDs Virtual Vision VCAP Sony Glasstron
  • 25. DigiLens Compact HOE C Solid state optics Switchable Bragg Grating Stacked SBG Fast switching Ultra compact www.digilens.com
  • 26. The Virtual Retinal Display Image scanned onto retina age sca e o to et a Commercialized through Microvision Nomad System - www.mvis.com
  • 27. Strengths of optical AR Simpler (cheaper) Direct view of real world Di i f l ld Full resolution, no time delay (for real world) Safety Lower distortion No eye displacement but COASTAR video see-through avoids this
  • 28. Video AR Architecture Head-mounted camera aligned to head position/orientation display optics Video image Head Tracker of real world Video Processor Host Graphics Digital Frame Processor P renderer d Mixer Mi Buffer to network Display Driver Virtual image inset into Non see-thru video of real Image source world & optics
  • 29. Video see-through HMD Video cameras Video Graphics Monitors Combiner
  • 31. Video see-through HMD MR Laboratory’s COAS A HMD ’ COASTAR (Co-Optical Axis See-Through Augmented Reality) Parallax-free Parallax free video see through HMD see-through
  • 32. TriVisio www.trivisio.com p Stereo video input PAL resolution cameras 2 x SVGA displays 30 degree FOV User adjustable convergence $6,000 USD
  • 33. Vuzix Display www.vuzix.com Wrap 920 $350 USD Twin 640 x 480 LCD displays 31 degree diagonal field of view Weighs less than three ounces
  • 34. Strengths of Video AR True occlusion Kiyokawa optical display that supports occlusion y p p y pp Digitized image of real world Flexibility Fl b l in composition Matchable time delays More registration, calibration strategies Wide FOV is easier to support
  • 35. Optical vs. Video AR Summary Both have proponents Video is more popular today? Likely because lack of available optical products y p p Depends on application? Manufacturing: optical i cheaper M f i i l is h Medical: video for calibration strategies
  • 36. Eye multiplexed AR Architecture head position/orientation Head Tracker real world Host Data Base Rendering Frame Processor P Model M d l Engine E i Buffer to network Display Virtual Image Driver inset into real world scene Opaque Image source
  • 39. Virtual image inset into real world
  • 41. Spatial Augmented Reality Project onto irregular surfaces Geometric Registration Projector blending, High dynamic range Book: Bimber, Rasker “Spatial Augmented Reality” p g y
  • 42. Projector-based AR User (possibly head-tracked) Projector Examples: Real objects Raskar, Raskar MIT Media Lab with retroreflective Inami, Tachi Lab, U. Tokyo covering
  • 43. Example of projector-based AR Ramesh Raskar, UNC, MERL , ,
  • 44. Example of projector-based AR Ramesh Raskar, UNC Chapel Hill
  • 45. The I/O Bulb Projector + C P j Camera John Underkoffler, Hiroshi Ishii MIT Media Lab
  • 46. Head Mounted Projector Head Mounted Projector J Jannick Rolland ( (UCF) ) Retro-reflective Material Potentially portable
  • 47. Head Mounted Projector NVIS P 50 HMPD P-50 1280x1024/eye Stereoscopic Stere sc ic 50 degree FOV www.nvis.com i
  • 48. HMD vs. HMPD Head Mounted Display Head Mounted Projected Display
  • 49. Pico Projectors Microvision - www.mvis.com 3M, Samsung, Phili etc 3M S Philips, t
  • 50. MIT Sixth Sense Body worn camera and projector p p y p j http://www.pranavmistry.com/projects/sixthsense/
  • 52. Video Monitor AR Video Stereo cameras Monitor g glasses Video Graphics Combiner
  • 53. Virtual Showcase Mirrors on a projection table Head H d tracked stereo k d Up to 4 users Merges graphic and real objects M hi d l bj Exhibit/museum applications Fraunhofer Institute (2001) Bimber, Frohlich
  • 54. Augmented Paleontology Bimber et. al. IEEE Computer Sept. 2002
  • 56. Handheld Displays Mobile Phones Camera Display Input
  • 57. Other Types of AR Display Audio spatial sound p ambient audio Tactile T til physical sensation Haptic virtual touch
  • 58. Haptic Input AR Haptic Workbench CSIRO 2003 – Adcock et al et. al.
  • 60. AR Haptic Interface Phantom, ARToolKit, Magellan g
  • 61. AR Tracking and Registration
  • 62. Tracking Locating the users viewpoint g p Position (x,y,z) Orientation (r p y) (r,p,y) Registration Positioning virtual object wrt real world
  • 63. Tracking Requirements Head Stabilized Body Stabilized World Stabilized Augmented Reality Information Display World Stabilized Body Stabilized Increasing Tracking Requirements Head Stabilized
  • 64. Tracking Technologies • Mechanical • g Electromagnetic • Optical • Acoustic • Inertial d dead I ti l and d d reckoning k i • GPS • Hybrid
  • 65. AR Tracking Taxonomy AR TRACKING Indoor Outdoor Environment Environment Limited Range Extended Range Low Accuracy & High Accuracy Not Robust & Robust Low Accuracy High Accuracy Many Fiducials Not Hybridized Hybrid Tracking at 15-60 Hz & High Speed in space/time GPS or GPS and Hybrid H b id but b t Camera or C Camera and C d Tracking no GPS Compass Compass e.g. e g AR Toolkit e.g. e g IVRD e.g. e g HiBall e.g. e g WLVA e.g. e g BARS
  • 66. Tracking Types Magnetic Inertial Ultrasonic Optical Mechanical Tracker T k Tracker T k Tracker T k Tracker T k Tracker Specialized Marker-Based Marker Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point g Tracking g Tracking g Tracking
  • 67. Tracking Systems Mechanical Tracker Magnetic Tracker Ultrasonic Tracker Inertial Tracker Vision (Optical Tracking) Specialized (Infrared, Retro-Reflective) Monocular (DVCam, Webcam) M l (DVC W b )
  • 68. Mechanical Tracker Idea: Id mechanical arms with joint sensors h l h Microscribe ++: high accuracy haptic feedback accuracy, -- : cumbersome, expensive
  • 69. Magnetic Tracker Idea: difference between a magnetic transmitter and a receiver Flock of Birds (Ascension) ++: 6DOF robust 6DOF, b -- : wired, sensible to metal, noisy, expensive y p
  • 70. Inertial Tracker I lT k Idea: measuring linear and angular orientation rates (accelerometer/gyroscope) IS300 (Intersense) Wii Remote ++: no transmitter, cheap small high frequency wireless transmitter cheap, small, frequency, -- : drift, hysteris only 3DOF
  • 71. Ultrasonics Tracker Idea: Time of Flight or Phase Coherence Sound Waves Phase-Coherence Ultrasonic Logitech IS600 ++: Small, Cheap -- : 3DOF, Line of Sight, Low resolution, Affected Environment Conditon (pressure, temperature)
  • 72. Global Positioning System (GPS) Created by US in 1978 Currently 29 satellites Satellites send position + time GPS Receiver positioning p g 4 satellites need to be visible Differential time of arrival Triangulation Accuracy y 5-30m+, blocked by weather, buildings etc
  • 73. Problems with GPS Takes time to get satellite fix Satellites moving around Earths atmosphere affects signal p g Assumes consistent speed (the speed of light). Delay depends where you are on Earth Weather effects Signal reflection Multi-path reflection off buildings Signal blocking Trees, buildings, mountains T b ildi t i Satellites send out bad data Misreport their own position
  • 74. Accurate to < 5cm close to base station (22m/100 km) Expensive - $20-40,000 USD
  • 76. Optical Tracker Idea: Image Processing and Computer Vision g g p Specialized Infrared, Retro-Reflective, Stereoscopic f f S ART Hi-Ball Monocular Based Vision Tracking g
  • 78. Optical Tracking Technologies Scalable active trackers InterSense IS-900, 3rd Tech HiBall 3rd Tech, I 3 d T h Inc. Passive optical computer vision Line of sight, may require landmarks sight Can be brittle. Computer vision i computationally-intensive C i i is i ll i i
  • 79. HiBall Tracking System (3rd Tech) Inside-Out Tracker O $50K USD Scalable over large area Fast d t (2000H ) F t update (2000Hz) Latency Less than 1 ms. Accurate Position 0.4mm RMS 0 4mm Orientation 0.02° RMS
  • 81. Starting simple: Marker tracking Has been done for more than 10 years Several open source solutions exist S l l i i Fairly simple to implement y p p Standard computer vision methods A rectangular marker provides 4 corner points l k id i Enough for pose estimation!
  • 82. Marker Based Tracking: ARToolKit http://artoolkit.sourceforge.net/
  • 84. Marker T k M k Tracking – O Overview
  • 85. Marker Tracking – Fiducial Detection Threshold the whole image to black and white Search scanline by scanline for edges (white to black) Follow edge until either Back to starting pixel Image border Check for size Reject fiducials early that are too small (or too large)
  • 86. Marker Tracking – Rectangle Fitting Start with an arbitrary point “x” on the contour S ih bi i “ ” h The point with maximum distance must be a corner c0 Create a diagonal through the center C d l h h h Find points c1 & c2 with maximum distance left and right of diag. New diagonal from c1 to c2 Find point c3 right of diagonal with maximum distance
  • 87. Marker Tracking – Pattern checking Calculate homography using the 4 corner points “Direct Linear Transform” algorithm Maps normalized coordinates to marker coordinates p (simple perspective projection, no camera model) Extract pattern by sampling Check pattern Id (implicit encoding) Template ( T l (normalized cross correlation) li d l i )
  • 88. Marker Tracking – Corner refinement Refine corner coordinates Critical for high quality tracking Remember: 4 points is the bare minimum! So these 4 points should better be accurate… Detect sub-pixel coordinates E.g. Harris corner detector g - Specialized methods can be faster and more accurate Strongly reduces jitter! gy j Undistort corner coordinates Remove radial distortion from lens R di l di t ti f l
  • 89. Marker M k tracking – P k Pose estimation Calculates marker position and rotation C relative to the camera Initial estimation directly from homography Very fast, but coarse fast Jitters a lot… Refinement via G R fi i Gauss-Newton iteration N i i 6 parameters (3 for position, 3 for rotation) to refine At each iteration we optimize on the reprojection error
  • 91. Coordinates for Marker Tracking •Camera Observed Screen Marker Ideal Screen Screen •Ideal ScreenCamera •Marker Observed •Perspective model (barrel shape) •Goal •Correspondence of •Nonlinear function 4 vertices •Obtained fromTranslation •Rotation & Camera Calibration •Obtained fromT processing R t ti •Real time image l ti Camera Calibration
  • 92. From Marker To Camera F M k T C Rotation & Translation TCM : 4x4 transformation matrix from marker coord. to camera coord.
  • 93. Tracking challenges in ARToolKit Occlusion Unfocused camera, Dark/unevenly lit Jittering (image by M. Fiala) motion blur scene, vignetting (Photoshop illustration) Image noise False positives and inter-marker confusion (e.g. poor lens, block coding / compression, neon tube) (image by M. Fiala)
  • 95. Other Marker Tracking Libraries arTag T http://www.artag.net/ ARToolKitPlus [Discontinued] http://studierstube.icg.tu- graz.ac.at/handheld_ar/artoolkitplus.php graz ac at/handheld ar/artoolkitplus php stbTracker http://studierstube.icg.tu- htt // t di t b i t graz.ac.at/handheld_ar/stbtracker.php MXRToolKit http://sourceforge.net/projects/mxrtoolkit/
  • 98. Markerless Tracking No more Markers! Markerless Tracking Magnetic T k M i Tracker Inertial I i l Ultrasonic Ul i Optical O i l Tracker Tracker Tracker Specialized Marker-Based Markerless Tracking Tracking Tracking Edge-Based Template-Based Interest Point Tracking Tracking Tracking
  • 99. Natural feature tracking Tracking from features of the surrounding environment Corners, edges, blobs, ... Generally more diffi l than marker tracking G ll difficult h k ki Markers are designed for their purpose g p p The natural environment is not… Less well established methods well-established Usually much slower than marker tracking
  • 100. Natural Feature Tracking Features Points Use Natural Cues of Real Elements Contours Edges Surface Texture Interest Points Model or Model-Free ++: no visual pollution Surfaces
  • 102. Tracking by d T k b detection This is what most trackers do do… Camera Image g Targets are detected every frame Keypoint detection yp Popular because tracking and detection Descriptor creation are solved simultaneously and matching d t hi Outlier Removal Pose estimation and refinement Pose
  • 103. Natural feature tracking – What is a keypoint? It depends on the detector you use! For high performance use the FAST corner detector Apply A l FAST t all pixels of your i to ll i l f image Obtain a set of keypoints for your image - R d Reduce the amount of corners using non-maximum suppression h f Describe the keypoints E. Rosten and T. Drummond (May 2006). "Machine learning for high‐speed corner detection". 
  • 105. Natural feature tracking – Descriptors Again depends on your choice of a descriptor! Can use SIFT Estimate the d i E i h dominant keypoint k i orientation using gradients Compensate for C m ensate f r detected orientation Describe the keypoints in terms of the gradients surrounding it Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D.,  Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones. IEEE Transactions on Visualization and Computer Graphics, May/June, 2010 
  • 106. NFT – D b Database creation Offline step p Searching for corners in a static image For robustness look at corners on multiple scales Some corners are more descriptive at larger or smaller scales We d ’t k W don’t know how far users will be from our image h f ill b f i Build a database file with all descriptors and their position on the original i ii h i i l image
  • 107. NFT – R l Real-time tracking k Search for keypoints Camera Image in the video image Create the d C t th descriptorsi t Keypoint detection Match the descriptors from the Descriptor creation p live video against those and matching in the database Outlier Removal O tli R l Brute force is not an option Need the speed-up of special Pose estimation data structures and refinement - E.g., we use multiple spill trees Pose
  • 108. NFT – O l removal Outlier l Cascade of removal techniques Start with cheapest, finish with most expensive… First simple geometric tests - E.g., line tests • Select 2 points to form a line • Check all other points being on correct side of line Then, homography-based tests
  • 109. NFT – P Pose refinement f Pose from homography makes good starting point Based on Gauss-Newton iteration Try to minimize the re-projection error of the keypoints Part of tracking pipeline that mostly benefits from floating point usage Can still be implemented effectively in fixed point Typically 2-4 iterations are enough…
  • 110. NFT – R l Real-time tracking k Search for keypoints Camera Image in the video image p Create the descriptors Keypoint detection Match the descriptors from the live video against those Descriptor creation and matching in the database Remove the keypoints that Outlier Removal are outliers Pose estimation Use h U the remaining k i i keypoints i and refinement to calculate the pose Pose of the camera f h
  • 111. NFT – R l Results Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D.,  Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones. IEEE Transactions on Visualization and Computer Graphics, May/June, 2010 
  • 112. Edge Based Tracking RAPiD [Drummond et al. 02] Initialization, Control Points, Pose Prediction (Global Method)
  • 113. Line Based Tracking Visual Servoing [Comport et al. 2004]
  • 114. Model Based Tracking OpenTL - www.opentl.org General purpose library for model based visual tracking
  • 117. Visual Modalities Used For Tracking
  • 119. Marker vs. natural feature tracking Marker tracking Usually requires no database to be stored Markers can be an eye-catcher Tracking is less demanding g g The environment must be instrumented with markers Markers usually work only when fully in view y y y Natural feature tracking A database of keypoints must be stored/downloaded Natural feature targets might catch the attention less Natural f t N t l feature targets are potentially everywhere t t t ti ll h Natural feature targets work also if partially in view
  • 121. Example: Outdoor Hybrid Tracking Combines computer vision - natural feature tracking inertial gyroscope sensors Both correct for each other Inertial gyro - provides frame to frame prediction of camera orientation Computer vision - correct for gyro drift
  • 122. Outdoor AR Tracking System You, Neumann, Azuma outdoor AR system (1999)
  • 123. Robust Outdoor Tracking Hybrid T ki H b id Tracking Computer Vision, GPS, inertial Going Out Reitmayer & Drummond (Univ. Cambridge)
  • 126. The Registration Problem Virtual and Real must stay properly aligned If not: t Breaks the illusion that the two coexist Prevents acceptance of many serious applications
  • 127. Sources of registration errors Static errors S Optical distortions Mechanical misalignments Tracker errors Incorrect viewing parameters Dynamic errors System delays (largest source of error) - 1 ms d l = 1/3 mm registration error delay i t ti
  • 128. Reducing static errors Distortion compensation Manual adjustments View-based or direct measurements [Azuma94] [Caudell92] [Janin93] etc. Camera calibration (video) [ARGOS94] [Bajura93] [Tuceryan95] etc.
  • 129. View Based Calibration (Azuma 94)
  • 130. Dynamic errors Application Loop x,y,z Tracking r,p,y Calculate Render Draw to Viewpoint Scene Display Simulation 20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms Total Delay = 50 + 2 + 33 + 17 = 102 ms 1 ms delay = 1/3 mm = 33mm error
  • 131. Reducing dynamic errors (1) Reduce system lag [Olano95] [Wloka95a] [Regan SIGGRAPH99] Reduce apparent lag Image deflection [Burbidge89] [Regan94] [So92] [Kijima [Kiji ISMR 2001] Image warping [Mark 3DI 97]
  • 132. Reducing System Lag Application Loop x,y,z Tracking r,p,y Calculate Render Draw to Viewpoint Scene Display Simulation Faster Tracker Faster CPU Faster GPU Faster Display
  • 133. Reducing Apparent Lag Virtual Display Virtual Display x,y,z y Physical Ph i l r,p,y Physical Display Display (640x480) (640x480) Tracking 1280 x 960 Update 1280 x 960 Last known position Latest position Application Loop x,y,z Tracking r,p,y Calculate Render Draw to Viewpoint p Scene Display p y Simulation
  • 134. Reducing dynamic errors (2) Match input streams (video) Delay video of real world to match system lag Predictive Tracking [Azuma94] [Emura94] Inertial sensors helpful Azuma / Bishop 1994 u a s op 99
  • 135. Predictive Tracking Position Now Past Future Time Can predict up to 80 ms in future (Holloway)
  • 137. Wrap-up Tracking and Registration are key problems Registration error Measures against static error Measures against dynamic error M i d i AR typically requires multiple tracking technologies yp y q p g g Research Areas: Hybrid Markerless Techniques, Deformable Surface, Mobile, Outdoors Surface Mobile
  • 138. More Information • M k Billi h t Mark Billinghurst – mark.billinghurst@hitlabnz.org • Websites – www.hitlabnz.org hi l b