Presentation on Mobile AR at the ICISTS conference in Korea, August 1st 2011

1. Going Out
Research in Mobile Augmented
Reality
Mark Billinghurst
HIT Lab NZ
University of Canterbury

3. Augmented Reality (Azuma 97)
 Combines Real and Virtual Images
- Both can be seen at the same time
 Interactive in real-time
- The virtual content can be interacted with
 Registered in 3D
- Virtual objects appear fixed in space

4. 1999: AR Face to Face Collaboration

5. Siggraph 99 Demo

6. 1998: SGI O2 2008: Nokia N95
$5,000 $500
CPU: 300 Mhz CPU: 332 Mhz
HDD; 9GB HDD; 8GB
RAM: 512 mb RAM: 128 mb
Camera: VGA 30fps Camera: VGA 30 fps
Graphics:500K poly/sec Graphics:2m poly/sec

7. Evolution of Mobile AR
Camera phone
Camera phone
- Thin client AR
Wearable Wearable AR
Computers Camera phone
- Self contained AR
PDAs
Handheld -Thin client AR
AR Displays
PDAs
-Self contained AR
1995 1997 2001 2003 2004

8. Mobile Phone AR
 Mobile Phones
 camera
 processor
 display
 AR on Mobile Phones
 Simple graphics
 Optimized computer vision
 Collaborative Interaction

9. Collaborative AR
 AR Tennis
 Two user game
 Audio + haptic
feedback
 Bluetooth messaging

10. AR Tennis

11. Mobile AR
 Txt message to download AR application (200K)
 See virtual content popping out of real paper advert
 Tested May 2007 by Saatchi and Saatchi

12. Mobile AR: Touring Machine (1997)
 University of Columbia
 Feiner, MacIntyre, Höllerer, Webster
 Combines
 See through head mounted display
 GPS tracking
 Orientation sensor
 Backpack PC (custom)
 Tablet input

13. MARS View
 Virtual tags overlaid on the real world
 “Information in place”

14. 2008 - Location Aware Phones
Motorola Droid Nokia Navigator

15. Real World Information Overlay
 Tag real world locations
 GPS + Compass input
 Overlay graphics data on live video
 Applications
 Travel guide, Advertising, etc
 Eg: Mobilizy Wikitude
 Android based, Public API released
 Other companies
 Layar, AcrossAir, Tochnidot, RobotVision, etc

16. Layar – www.layar.com
iPhone, Android
> 2 million downloads
 1500+ information layers

20. HIT Lab NZ Outdoor AR Platform
 Cross platform
 Android, iPhone
 3D onsite visualization
 Intuitive user interface
 Positions content in space
 Camera, GPS, compass
 Client/Server software architecture
 Targeting museum guide/outdoor site
applications

21. Prototype: Earthquake Reconstruction
 See past, present and future building designs
 Earthquake survivor stories shown on map
view
 Collect user comments
 Android platform

22. Benefits of Mobile AR
 Making the invisible visible
 Geo-located services/information
 Ease interaction with virtual content
 Tangible/physical interaction
 Reduce cognitive load
 Intuitive spatial organization of information

23. Mobile AR Today
 Smart Phones (450+ million in 2011)
 Cameras, GPS, Compass, Gyroscope
Sensors
 Fast CPU, GPU chips
 Free developer kits
 Layar,Junaio, outdoor SDKs
 Qualcomm QCAR vision tracking SDK
 User-generated content
 Flickr, Sketch-up, YouTube, Twitter, etc

24. $784 million USD in 2014

25. Looking to the Future

26. Directions for Future Research
 Enabling Technologies
 Displays, tracking, information filtering
 User Experience
 Remove social boundaries, increasing ease of
use
 Crossing Boundaries
 AR + other interface metaphors
 Social Augmented Reality
 AR 2.0, AR everywhere

27. Tri-corder Problem
 Handheld AR
 Social boundaries
 Not always available

28. Future Displays
 Always on, unobtrusive

29. Contact Lens Display
 Babak Parviz
 University Washington
 MEMS components
 Transparent elements
 Micro-sensors
 Challenges
 Miniaturization
 Assembly
 Eye-safe

30. AR User Experience

33. Crossing Boundaries
Jun Rekimoto, Sony CSL

34. Invisible Interfaces
Jun Rekimoto, Sony CSL

35. The MagicBook
Reality Augmented Augmented Virtuality
Reality (AR) Virtuality (AV)

36. ubiHome @ GIST
Media services Light service MR window
ubiTrack
Where/When
Tag-it
ubiKey
ubiHome
Who/What/
What/When/How When/How
PDA Couch Sensor Door Sensor
Who/What/When/How When/How When/How

37. CAMAR - GIST
(CAMAR: Context-Aware Mobile Augmented Reality)

38. Ubiquitous AR (GIST, Korea)
 How does your AR device work with other devices?
 How is content delivered?

39. Ubiquitous
UbiComp
Ubi AR
Ubi VR
Weiser Mobile AR
Desktop AR VR
Terminal
Reality Virtual Reality
Milgram
From: Joe Newman

40. Massive
Multi User
Ubiquitous
Terminal
Single User
Reality
VR

41. Multi-user AR Interfaces
1996 1999 2004 2018
2 users 4 users 8 users ?? users
 10 years of Collaborative AR – 2 to 8 users
 cf MSN Chat 29m, Skype 17m, Second Life 50K
 Mobile/handheld AR scales to high number of users
 New applications, infrastructure, content distribution…

42. Social Augmented Reality
 Public and private annotations everywhere

43. BASIC VIEW

44. PERSONAL VIEW

45. Augmented Reality 2.0
Infrastructure

46. Social Implications
 Privacy
 Who can see my information?
 Reliability (Wikipedia effect)
 How truthful is the information?
 Just in time education
 Information without learning?
 Attention Becomes a Valuable Commodity
 Hyperconnectivity = time slicing

47. Conclusions
• Mobile AR enhances interaction with real
world
• Many possible applications/commercial
possibilities
• Important research problems need to be
solved
–Enabling Technologies
–Experience Design
–Crossing Boundaries
–Social Augmented Reality

48. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
• Website
– www.hitlabnz.org