Save 10% off ANY FITC event with discount code 'slideshare' See our upcoming events at www.fitc.ca OVERVIEW Developing Interactive 3D Experiences in HTML5 with Carlos Ulloa Modern web browsers are capable of producing some amazing audiovisual candy, through new capabilities like HTML5, fast JavaScript, WebGL and Web Audio API. In this session, Carlos talks in detail about his work developing interactive 3D experiences using these technologies. He will take an in-depth look at the production of high-end WebGL projects including Lights, HelloRun and Racer-S, to discuss the graphics, audio and input development techniques used. He shows how to take advantage of the three.js engine to deliver immersive 3D visuals running at high performance on desktop and mobile browsers. This presentation will also cover music visualization and sound effects, using Web Audio API for sound processing and analysis. Carlos also discusses the integration of a broad range of input technologies like touch, mouse, keyboard, accelerometer and Leap Motion controller, along with some interesting lessons learned.

1. Developing Interactive
3D Experiences in HTML5
Carlos Ulloa
HelloEnjoy.com
FITC Amsterdam 2014

2. @C4RL05

10. GFX
AUDIO
INPUT

11. Developing
Interactive 3D
Experiences in
HTML5

14. 1-CLICK

15. WORK
FLOW

16. Workflow first
& then performance

17. Test early
First build in 2 weeks

18. Test often
One or two builds per week

19. Prototype
From concept & design

20. GFX

21. GAME LOOP

22. Optimise
your game loop

23. Object Pooling
Minimise garbage collection

24. Spread
& interpolate over frames

25. TIME

26. Frame Rate
Independent

27. requestAnimationFrame

28. deltaTime
Since the last frame

29. Multiply by
deltaTime
to update values
per second
instead of per frame

30. Cap deltaTime
< 0.1 second

31. timeScale
Multiplies deltaTime

32. Keep Sync
with music & video

33. THREE.JS

34. Examples
threejs.org/examples

35. Source Code
three.js/src

36. Docs
threejs.org/docs

37. ASSET PIPELINE

42. MATH

43. VEC
QUAT
MTRX

44. VECTORS

45. Vectors
Represent position, direction,
velocity, acceleration, forces…

46. ( x, y, z )
Not very intuitive on multiple axes

47. object.position
Local position

48. object.scale
Local scale

49. |a|
Magnitude

50. |a| = 1
Normalized

51. +

52. DOT PRODUCT

55. Dot Product

56. |a| |b| cos
Dot Product

57. +
CROSS PRODUCT

60. Cross Product

61. |a| |b| sin
Cross Product

62. QUATERNIONS

63. Quaternions
Represent rotations

64. object.quaternion
Local rotation

65. Multiply quaternions
to rotate a vector or another rotation

66. Slerp quaternions
to interpolate rotations

67. Avoid Euler
on multiple axes

68. MATRICES

69. Matrices
Represent transformations
i.e. position, rotation, scale

70. object.matrix
Local transform

71. object.matrixWorld
Global transform

72. Multiply matrices
to transform a vector or another matrix

73. Transform methods
lookAt, translateXYZ, rotateXYZ,
localToWorld, worldToLocal

74. OBJECTS

75. HIERARCHY

76. GROUPS

77. PIVOTS

78. Children methods
getDescendants, traverse

79. Helper Objects
Arrow, Axis, Bounding Box,
Camera, Grid, Lights

80. MESHES

81. Geometry
Build your own

82. Vertex Color
Paint, bake or create in code

84. TEXTURES

85. Mip Maps
Smaller versions of the texture

86. Offset & Repeat
Transform textures

87. Anisotropy
Fix mip map artifacts

88. Wrap
Repeat, Clamp, Mirror

89. Clamp Wrap
Fix artifacts on texture seams

90. RGBA
Use all channels

92. LIGHTS

93. Dynamic Lights
Real-time surface illumination

94. Baked Lights
Better quality & performance

95. Lightmaps
Global illumination

96. Vertex Bake
Quality depends on geometry

97. CAMERA

98. Avoid
default camera settings

99. Near Plane
Occlusion & fill rate performance

100. Far Plane
Horizon pop up & removed geometry

101. FRUSTUM
CULLING

102. Near
Far

105. Bounding
Sphere

109. FOV
Vertical angle

110. FOV

111. FOV
Different for each scene

112. POST PROCESSING

113. FXAA

114. 8X

115. 8X

116. SSAO

117. Cry Engine

118. GLOW

120. VIGNETTE

123. OPTIMISE

124. Profile & measure
Understand what’s going on

125. GPU
CPU

126. GPU

127. GPU?
Faster in lower res

128. GPU
Many pixels, fill rate bound

129. CPU

130. CPU?
Faster with objects disabled

131. CPU
Many objects, many draw calls

132. DRAW CALLS

133. Minimise
state changes & draw calls

134. Batching
Combine meshes with same material

135. Batch Static
Always combine static objects

136. Combine Materials
Use vertex & textures to differentiate

137. Texture Atlas
Combine textures

139. Multiple Passes
Multiple draw calls

140. SHADERS

141. Objects < Vertices < Pixels

142. Vertex
Instead of fragment

143. Precission
Colors & normals

144. Alpha Test
Instead of Alpha Blend

145. Fragment Math
Avoid pow, sin, cos

146. Vector Math
Dot & Cross Product

147. LOD

148. Level of detail
Simpler far away objects

149. Optimised
geometries or materials

150. Sprites
& particle systems

151. Skybox & Rings
for very distant objects

152. MOBILE

153. Mobile < Desktop

154. Low Polycount
> 500

155. Overdraw
Depends on the GPU

156. Fill Rate
devicePixelRatio

157. Upscale
For higher resolutions

158. Downscale
For higher quality renders

159. Simple Shaders
Minimise fragment complexity

160. No Bumps
Avoid normal mapping

161. No Lights
Avoid dynamic lighting or shadows

162. No Fog
Only on vertex shader

163. No Post
Avoid full screen effects

164. lowp
getShaderPrecisionFormat

165. AUDIO

166. WEB AUDIO

167. Audio
Processing & analysis

168. Audio Node
Web Audio API building block

171. Panner Node
for positional audio

172. Logarithmic
scale for spectrum analysis

174. INPUT

175. KEYBOARD

176. Analog
Simulate key pressure

177. Each key
on its own

178. Combine
keys to create axes

179. MOUSE

180. Pointer Lock
Relative mouse movement

181. Relative Mouse
Accumulated deltas

182. DPI
& different screen sizes

183. Sensitivity
Users ignore settings

184. TOUCH

185. Relative
touch controls

186. Orientation
How users hold the device?

187. Secondary
touch support

188. LEAP MOTION

189. Smooth
Introduces lag

190. Calibration
Don’t leave it to the user

191. Relative
Real time calibration

192. Slides
helloenjoy.com/docs/3DJSDev.pdf

193. DANK U
helloenjoy.com