The first lecture from the Augmented Reality Summer School talk by Mark Billinghurst at the University of South Australia, February 15th - 19th, 2016. This provides an introduction to Augmented Reality and overview of the history.

1. LECTURE 1: INTRODUCTION
TO AUGMENTED REALITY
Mark Billinghurst
AR Summer School
February 15th – 19th 2016
University of South Australia

2. A Brief History ofTime
• Trend
• smaller, cheaper, more functions, more intimate
• Technology becomes invisible
• Intuitive to use
• Interface over internals
• Form more important than function
• Human centered design

3. A Brief History of Computing
• Trend
• smaller, cheaper, faster, more intimate, intelligent objects
• Computers need to become invisible
• hide the computer in the real world
• Ubiquitous / Tangible Computing
• put the user inside the computer
• Virtual Reality

4. Making Interfaces Invisible
Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented
interaction with real world environments. In Proceedings of the 8th Annual ACM Symposium on
User interface and Software Technology. UIST '95. ACM, New York, NY, 29-36.

5. Graphical User Interfaces
• Separation between real and digital worlds
• WIMP (Windows, Icons, Menus, Pointer) metaphor

6. Ubiquitous Computing
• Computing and sensing embedded in real world
• Particle devices, RFID, motes, arduino, etc

7. Virtual Reality
• 1985…

8. Virtual Reality
• ImmersiveVR
• Head mounted display, gloves
• Separation from the real world

9. Occulus Rift
• $300 USD
• 360 degree head tracking
• 100 degree field of view

10. 1977 – StarWars

11. Augmented Reality Definition
• Defining Characteristics [Azuma 97]
• Combines Real andVirtual Images
• Both can be seen at the same time
• Interactive in real-time
• The virtual content can be interacted with
• Registered in 3D
• Virtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.

12. 2008 - CNN

13. • Put AR pictures here
Augmented Reality Examples

14. AR vsVR

15. Making Interfaces Invisible
Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented
interaction with real world environments. In Proceedings of the 8th Annual ACM Symposium on
User interface and Software Technology. UIST '95. ACM, New York, NY, 29-36.

16. From Reality toVirtual Reality
Ubiquitous Computing Augmented Reality Virtual Reality

17. Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays
IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.

18. AugmentedVirtuality
• VR with windows into the real world

19. Ubiquitous Computing Continuum
• Weiser Continuum

20. Milgram – Weiser Continuum
• Newman, J., Bornik, A., Pustka, D., Echtler, F., Huber, M., Schmalstieg, D., & Klinker, G. (2007, March). Tracking for
distributed mixed reality environments. In Workshop on Trends and Issues in Tracking for Virtual Environments at the IEEE
Virtual Reality Conference (VR’07).

21. Metaverse
• Neal Stephenson’s “SnowCrash”
• The Metaverse is the convergence of:
• 1) virtually enhanced physical reality
• 2) physically persistent virtual space
• Metaverse Roadmap
• http://metaverseroadmap.org/

22. Metaverse Dimensions
• Augmentation technologies that layer information
onto our perception of the physical environment.
• Simulation refers to technologies that model
reality
• Intimate technologies are focused inwardly, on the
identity and actions of the individual or object;
• External technologies are focused outwardly,
towards the world at large;

23. Metaverse Components
• Four Key Components
• Virtual Worlds
• Augmented Reality
• Mirror Worlds
• Lifelogging

25. MirrorWorlds
• Mirror worlds are informationally-enhanced virtual
models of the physical world.
• Google Earth, MS StreetView, Google Maps

27. LifeLogging
• Technologies record and report the intimate
states and life histories of objects and users
• Nokia LifeBlog, Nike+, FitBit

28. • Steve Mann - LifeLogging

29. Gordon Bell:LifeLogging
1 TB to store 65 years of data

30. Narrative Clip
• Wearable camera
• Automatic picture capture - 2 pics/minute
• http://getnarrative.com

31. Summary
• Augmented Reality has three key features
• Combines Real andVirtual Images
• Interactive in real-time
• Registered in 3D
• AR can be classified alongside other technologies
• Invisible Interfaces
• Milgram’s Mixed Reality continuum
• MetaVerse

32. AR History

33. Pepper’s Ghost (1862)
• Dates back to Giambattista della Porta (1584)

34. The Master Key (1901) – AR Glass
• "It consists of this pair of spectacles.
While you wear them every one you
meet will be marked upon the
forehead with a letter indicating his or
her character. The good will bear the
letter 'G,' the evil the letter 'E.' …
Thus you may determine by a single
look the true natures of all those you
encounter.”
L. Frank Baum

35. Early HUD (1958)
F16 – Head Up Display

36. Early HMD Patents

37. First HMD
• Philco Headsight (1961) – Remote Camera Viewing

38. Sutherland HMD
• 1960’s: Sutherland / Sproull’s
first HMD system was see-
through

39. USAirforce HMDs
1960 - 70’s: US Air Force helmet mounted
displays (T. Furness)

40. SuperCockpit Program
1970 - 80’s: US Air Force Super Cockpit (T. Furness)

41. Modern Airforce HMDs
• Honeywell Integrated Helmet and Display Sighting System
(IHADSS) deployed in 1985 on the AH-64 Apache attack helicopter

42. First Industrial Use
• Early 1990’s: Boeing coined the term “AR.” Wire harness
assembly application begun (T. Caudell, D. Mizell).

43. Academic Research
• 1994: Motion stabilized display [Azuma]
• 1995: Fiducial tracking in video see-through [Bajura / Neumann]
• 1996: UNC hybrid magnetic-vision tracker

44. Development of the Field
• 1996: MIT Wearable Computing efforts
• 1998: Dedicated conferences begin (ISMAR)
• Late 90’s: Collaboration, outdoor, interaction
• Late 90’s:Augmented sports broadcasts

45. History Summary
• 1960’s – 80’s: Early Experimentation
• 1980’s – 90’s: Basic Research
• Tracking, displays
• 1995 – 2005:Tools/Applications
• Interaction, usability, theory
• 2005 - : Commercial Applications
• Games, Medical, Industry

46. Important Milestones
• 1992:T. Caudell coined term “AR” (Boeing)
• 1996: First collaborative AR systems
• 1999:ARToolKit released – tracking library
• 1997: Feiner First outdoor AR system
• 1997 – 2001: Mixed Reality Systems Lab (Japan)
• 1999 – 2003:ARVIKA project (Germany)
• 2002: ISMAR conference started

47. 2007 -AR Reaches Mainstream
• MIT Technology Review
• March 2007
• list of the 10 most exciting
technologies
• Economist
• Dec 6th 2007
• Reality, only better

48. 2009 -AR in Magazines
• Esquire Magazine
• Dec 2009 issue
• 12 pages AR content
• Many Others
• Wired
• Colors
• Red Bull
• Etc

49. Google Searches forAR

50. 2008 - Browser BasedAR
• Flash + camera + 3D graphics
• High impact
• High marketing value
• Large potential install base
• 1.6 Billion web users
• Ease of development
• Lots of developers, mature tools
• Low cost of entry
• Browser, web camera

53. Impact ofWeb-basedAR
• Boffswana Living Sasquatch
• http://www.boffswana.com/news/?p=605
• In first month
• 100K unique visits
• 500K page views
• 6 minutes on page

54. 2005 - Mobile PhoneAR
• Mobile Phones
• camera
• processor
• display
• AR on Mobile Phones
• Simple graphics
• Optimized computer vision
• Collaborative Interaction

55. ARAdvertising (HIT Lab NZ 2007)
• Txt message to download AR application (200K)
• See virtual content popping out of real paper advert
• Tested May 2007 by Saatchi and Saatchi

56. 2008:LocationAware Phones
Nokia NavigatorMotorola Droid

57. 2009 - Outdoor Information Overlay
• Mobile phone based
• Tag real world locations
• GPS + Compass input
• Overlay graphics data on live video
• Applications
• Travel guide,Advertising, etc
• Wikitude, Layar, Junaio, etc..
• Android based, Public API released

58. Layar (www.layar.com)
• Location based data
• GPS + compass location
• Map + camera view
• AR Layers on real world
• Customized data
• Audio, 3D, 2D content
• Easy authoring
• Android, iPhone

59. Google Glass (2011 - )

60. Epson Moverio BT-200
▪ Stereo see-through display ($700)
▪ 960 x 540 pixels, 23 degree FOV, 60Hz, 88g
▪ Android Powered, separate controller
▪ VGA camera, GPS, gyro, accelerometer

61. ARToday
• Key Technologies Available
• Robust tracking (ComputerVision, GPS/sensors)
• Display (Handheld, HMDs)
• Input Devices (Kinect, etc)
• Developer tools (PTC,ARTW)
• Commercial Business Growing
• Gaming, GPS/Mobile, Online Advertisement
• >$600 Million USD in 2015

62. • Web based AR
• Flash, HTML 5 based AR
• Marketing, education
• Outdoor Mobile AR
• GPS, compass tracking
• Viewing Points of Interest in real world
• Eg: Junaio, Layar, Wikitude
• Handheld AR
• Vision based tracking
• Marketing, gaming
• Location Based Experiences
• HMD, fixed screens
• Museums, point of sale, advertising
Typical AR Experiences

63. AR BusinessToday
• Marketing
• Web-based, mobile
• Mobile AR
• Geo-located information and service
• Driving demand for high end phones
• Gaming
• Mobile, Physical input (Kinect, PS Move)
• Upcoming areas
• Manufacturing, Medical, Military

64. Some CommercialAR Companies
• ARToolworks (http://www.artoolworks.com/)
• ARToolKit, FLARToolKit, SDKs
• Total Immersion (http://www.t-immersion.com/)
• Marketing,Theme Parks,AR Experiences
• PTC (http://www.vuforia.com/)
• Mobile AR,Vuforia SDK
• Many small start-ups

65. Gartner Hype Cycle (2015)

66. AR Commercial Landscape

67. Market Forecast
Up to $120B by 2020 – 5 xVR market

68. User Forecast

69. Summary
• Augmented Reality has a long history going
back to the 1960’s
• Interest in AR has exploded over the last few
years and is being commercialized quickly
• AR is growing in a number of areas
• Mobile AR
• Web based AR
• Marketing experiences

70. Sample AR Applications

71. Applications
• Medicine
• Manufacturing
• Information overlay
• Architecture
• Museum
• Marketing
• Gaming

72. Applications:medical
• “X-ray vision” for surgeons
• Aid visualization, minimally-invasive operations. Training. MRI, CT
data.
• Ultrasound project, UNC Chapel Hill.
Courtesy
UNC
Chapel
Hill

73. MedicalARTrials
! Sauer et al. 2000 at
Siemens Corporate
Research, NJ
! Stereo video see through
F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a
Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial,
MICCAI 2002

74. Assembly and Maintenance
© 1993 S. Feiner, B. MacIntyre, &
D. Seligmann, Columbia University
© 1996 S. Feiner, B. MacIntyre, &
A. Webster, Columbia University

75. PS3 - Eye of Judgment (2007)
• Computer Vision Tracking
• Card based battle game
• Collaborative AR
• October 24th 2007

77. AR Books – MarkerlessTracking

78. ARAnnotations
Columbia
University
HRL
© 1993 S. Feiner, B. MacIntyre,
M. Haupt, & E. Solomon,
Columbia University
© 1997 S. Feiner, B. MacIntyre,
T. Höllerer, & A. Webster,
Columbia University

79. BroadcastTV

80. Interactive Museum Experiences
" BlackMagic
" Virtual America’s Cup
" 410,000 people in six months
" MagicPlanet
" TeManawa science museum
" Virtual Astronomy
" Collaborative AR experience
" ARVolcano
" Interactive AR kiosk
" Scienceworks museum, Melbourne

81. Digital Binocular Station
http://www.DigitalBinocularStation.com/

82. MuseumArcheology
• LifePlus (2002-2004)
• Natural feature tracking
• Virtual characters
• Mobile AR system
• Archeoguide (2000-2002)
• Cultural heritage on-site guide
• Hybrid tracking
• Virtual overlay

83. CityViewARApplication
• Visualize Christchurch before the earthquakes

84. Sales and Marketing
• Connect with brands and branded objects
• Location Based Experiences
• Lynx Angels
• Web based
• Rayban glasses
• Mobile
• Ford Ka campaign
• Print based
• Red Bull Magazine

85. Summary
• AR technology can be used to develop a wide
range of applications
• Promising application areas include
• Games
• Education
• Engineering
• Medicine
• Museums
• Etc..

86. www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au