If 2017 was the year of Virtual Reality, 2018 may become the year of Augmented Reality. With Apple's ARKit for iOS and Google's ARCore for Android, we should not forget that actually the open web and Augmented Reality are the perfect couple. But how do we create AR apps with web technology? In my talk you will learn how to write your own AR app that runs in the browser. You will see how Augmented Reality works, what markers are for, and how to create them. And you'll get an overview about the devices you can use to test your AR projects. We are just at the beginning of the rise of Augmented Reality. This is why lots of UX design questions are still unanswered. But there are patterns and best practises, and you will learn how to use them to get a great user experience in your AR application.

1. @fischaelameergeildanke.com @SCRIPT18
Augmented Reality
with Web Technology

2. AR

3. VR AR MR

4. Virtual Reality

5. Virtual Reality
completely digital

6. Augmented Reality virtual overlay

7. Mixed Reality virtual and real world coincide

8. @fischaelameergeildanke.com @SCRIPT18
A Short History
of Augmented Reality

9. Ivan Sutherland

10. Ivan Sutherland’s
The Sword of Damocles

11. Ivan Sutherland’s
The Sword of Damocles

12. Sword of Damocles, 1812
Richard Westall

13. https://www.youtube.com/watch?v=ISJWZpFIAlQ

14. Tom Caudell

16. ❝ Augmented Reality (AR) is a variation of
Virtual Environments (VE), or Virtual Reality as
it is more commonly called. ❞
Ronald Azura, 1997

17. ❝ VE technologies completely immerse
a user inside a synthetic environment. While
immersed, the user cannot see the
real world around him. ❞
Ronald Azura, 1997

18. ❝ In contrast, AR allows the user to see the
real world, with virtual objects superimposed
upon or composited with the real world.
Therefore, AR supplements reality, rather than
completely replacing it. ❞
Ronald Azura, 1997

19. The Touring Machine

20. @fischaelameergeildanke.com @SCRIPT18
Technology and Use Cases
of Augmented Reality Today

22. Innovation Trigger

23. Innovation Trigger
Peak of Inflated
Expectations

24. Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment

25. Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of
Enlightenment

26. Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of
Enlightenment
Plateau of
Productivity

27. Innovation Trigger
Peak of Inflated
Expectations
Trough of
Disillusionment
Slope of
Enlightenment
Plateau of
Productivity
Machine Learning
Blockchain
Augmented Reality
Virtual Reality

28. Camera
Tracking
GPS
Screen

29. PositionRotation

30. https://www.microsoft.com/de-de/hololens

31. http://www.metavision.com/

32. https://www3.lenovo.com/at/de/tango/

33. https://developer.apple.com/arkit/

34. https://developers.google.com/ar/

35. https://play.google.com/store/apps/details?id=com.evogames.stackitar

36. https://play.google.com/store/apps/details?id=com.roomle.android

37. https://play.google.com/store/apps/details?id=com.signalgarden.atomvisualizer

38. https://www.volkswagenag.com/de/group/research/virtual-technologies.html

39. https://www.linkedin.com/pulse/20140502184728-1282856-guide-to-augmented-reality-in-healthcare/

41. https://vrodo.de/augmented-reality-mit-hololens-microsoft-zeigt-holoportation/

43. https://techcrunch.com/2015/07/29/google-translates-app-now-instantly-translates-printed-text-in-27-languages/

44. https://www.urdesignmag.com/technology/2015/01/26/rideon-augmented-reality-ski-goggles/

45. @fischaelameergeildanke.com @SCRIPT18
Augmented Reality
and the Web?

46. Write Once, Runs Anywhere

47. Write Once, Runs Anywhere
No App Download Required

48. Camera
& Display

49. Camera
& Display
Tracking
Sensors

50. Camera
& Display
Tracking
Sensors
World
Recognition

51. WebGL API & WebVR API

52. WebGL API & WebVR API
Media Capture and Streams API

53. Marker-less AR
vs.
Marker-based AR

54. https://aframe.io/

55. @fischaelameergeildanke.com @SCRIPT18
Markerless Augmented Reality
with ARCore and ARKit

56. ARKit
by Apple

57. ARKit ARCore
by Apple by Google

58. ARKit ARCore
by Apple by Google
World/Motion Tracking Scene Analyzation Light Estimation

59. ARKit ARCore
by Apple by Google
WebAROnARKit WebAROnARCore
Two Experimental Browser Apps
https://github.com/google-ar/WebARonARKit https://github.com/google-ar/WebARonARCore

60. ARKit ARCore
by Apple by Google
three.ar.js
three.js for AR

62. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

63. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

64. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

65. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

66. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

67. https://furniture.3d.io/

68. <script src="aframe-master.min.js"></script>
<script src="three.ar.js"></script>
<script src="aframe-ar.js"></script>
<script src="3dio.min.js"></script>
<script src="main.js"></script>

69. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

70. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

71. let scene = new THREE.Scene();
let camera = new THREE.PerspectiveCamera(75,
window.innerWidth / window.innerHeight, 0.1, 10000);
let renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
function render() {
requestAnimationFrame(render);
renderer.render(scene, camera);
}
render();

72. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

73. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

74. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

75. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

76. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

78. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

79. <body>
<a-scene ar>
<a-entity io3d-furniture="id:123" visible="false"></a-entity>
<a-sphere id="intersection" visible="false"
radius="0.04" color="yellow"></a-sphere>
<a-camera>
<a-entity cursor="fuse:false" raycaster ar-raycaster></a-entity>
<a-entity position="0 0 -0.1" scale="0.001 0.001 0.001"
geometry="primitive: ring; radiusInner: 0.8; radiusOuter: 1"
material="color: yellow; shader: flat; transparent:true">
</a-entity>
</a-camera>
</a-scene>
</body>

80. let scene = document.querySelector('a-scene');
let raycaster = document.querySelector('[ar-raycaster]');
let mark = document.querySelector('#intersection');
raycaster.addEventListener('raycaster-intersection', function(e) {
mark.setAttribute('position', e.detail.intersections[0].point);
mark.setAttribute('visible', true);
});
raycaster.addEventListener('raycaster-intersection-cleared', function(e) {
mark.setAttribute('visible', false);
});

81. window.addEventListener('click', function () {
let furniture = document.querySelector('[io3d-furniture]');
let position = mark.getAttribute('position');
furniture.setAttribute('position', position);
furniture.setAttribute('visible', true);
});

83. https://3d.io/

84. @fischaelameergeildanke.com @SCRIPT18
Marker-based AR
with AR.js

85. https://poly.google.com/view/77wHkzwlpOq

86. <script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>

87. <script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>

88. <script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>

89. <script src="aframe.min.js"></script>
<script src="aframe-ar.js"></script>
<script src="annyang.min.js"></script>
<script src="aframe-speech-command-component.js"></script>

90. <a-scene arjs="trackingMethod:best;">
<a-assets>[…]</a-assets>
<a-anchor hit-testing-enabled="true">[…]</a-anchor>
<a-camera-static/>
</a-scene>

91. <a-assets>
<a-asset-item id="sun-obj" src="assets/sun.obj"></a-asset-item>
<a-asset-item id="sun-mtl" src="assets/sun.mtl"></a-asset-item>
</a-assets>

92. <a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

93. <a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

94. <a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

95. <a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

96. <a-entity id="annyang" annyang-speech-recognition></a-entity>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

97. <a-entity id="annyang" annyang-speech-recognition></a-entity>
<a-anchor hit-testing-enabled="true">
<a-box position="0 0.5 0" material="opacity:0;">
<a-entity obj-model="obj:#sun-obj; mtl:#sun-mtl"
scale="0.2 0.2 0.2" visible="false"
speech-command__show="command: sun;
type: attribute; attribute: visible; value: true;"
speech-command__hide="command: hide;
type: attribute; attribute: visible; value: false;">
<a-animation attribute="rotation" to="360 0 0"
dur="5000" easing="linear" repeat="indefinite">
</a-animation>
</a-entity>
</a-box>
</a-anchor>

98. https://poly.google.com/view/77wHkzwlpOq

99. https://jeromeetienne.github.io/AR.js/three.js/examples/arcode.html

100. @fischaelameergeildanke.com @SCRIPT18
UX Design for
Augmented Reality

101. @fischaelameergeildanke.com @SCRIPT18
No Established Patterns Yet

102. Interpretability
Comfort
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc

103. Interpretability
Comfort
Safety
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc

104. User Input and Interaction
User Interface and Information
Interpretability
Context
Comfort
Safety
Illusion
Usefulness
Delight
https://medium.com/@beaucronin/the-hierarchy-of-needs-in-virtual-reality-development-4333a4833acc

105. VoiceGestures Gaze Movement

106. Respond Immediately
User Input and
Interaction

107. Respond Immediately
Use Simple Interactions
User Input and
Interaction

109. Respond Immediately
Use Simple Interactions
Don’t Apply 2D patterns
User Input and
Interaction

110. User Interface
and Information
Provide Information in Context

111. DoDon’t

112. Provide Information in Context
Use Visual Indicators
User Interface
and Information

113. https://developer.apple.com/ios/human-interface-guidelines/technologies/augmented-reality/

114. Provide Information in Context
Use Visual Indicators
Onboarding is Important
User Interface
and Information

115. Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting

116. Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting

117. Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting

118. Walktrough
Camera Access Request
Scanning
Placing Objects
Interacting

119. DoDon’t
https://medium.com/inborn-experience/onboarding-in-augmented-reality-mobile-application-6e08e1d338f8

120. Provide Information in Context
Use Visual Indicators
Onboarding is Important
Error Handling, too
User Interface
and Information

121. Creating an
Illusion
Light and Shadow are Important

122. https://www.youtube.com/watch?v=ttdPqly4OF8

123. Light and Shadow are Important
Avoid Interruptions
Creating an
Illusion

124. Light and Shadow are Important
Avoid Interruptions
Avoid Clutter
Creating an
Illusion

125. Consider
the Context
Consider a User’s Environment

126. Consider a User’s Environment
There’s No Optimal Setting
Consider
the Context

127. Comfort
and Safety

129. Avoid Fatiguing The User
Comfort
and Safety

130. Avoid Fatiguing The User
Support Short Usage
Comfort
and Safety

131. Avoid Fatiguing The User
Support Short Usage
Avoid Large Movements
Comfort
and Safety

132. Avoid Fatiguing The User
Support Short Usage
Avoid Large Movements
Respect Safe Space
Comfort
and Safety

133. @fischaelameergeildanke.com @SCRIPT18
The Future of AR
in the Web

134. WebXR as an Extension
The Future of
AR in the Web

135. WebXR as an Extension
Declarative WebGL
The Future of
AR in the Web

136. WebXR as an Extension
Declarative WebGL
3D Positions for Elements
The Future of
AR in the Web

137. https://www.youtube.com/watch?v=8GXUlijX60E

138. Get involved!
The Future of
AR in the Web

139. @fischaelameergeildanke.com @SCRIPT18
Thank you!

140. https://github.com/jeromeetienne/AR.js/
https://github.com/google-ar/three.ar.js/
https://research.mozilla.org/mixed-reality/
https://github.com/mozilla/webxr-api
https://webxr.io/webar-playground/app/
Libraries
https://github.com/mozilla-mobile/webxr-ios
https://github.com/google-ar/WebARonARKit
https://github.com/google-ar/WebARonARCore
Browsers
https://3d.io/docs/api/1/aframe-components.html
https://3d.io/docs/api/1/furniture.html
https://poly.google.com/
https://sketchfab.com/
https://jeromeetienne.github.io/AR.js/three.js/examples/arcode.html
Assets