Web browsing on Extended Reality enables users to navigate and interact with traditional Web-based content, as well as immersive WebXR experiences. In this presentation, we will draw from our experience creating Wolvic, an open-source Web browser for XR devices. We will begin with an overview of current Web-based solutions for different use cases, from video and gaming to social collaboration and productivity. We will discuss the technical challenges and design opportunities of creating a Web browser for XR devices. Finally, we will look at the complexities of developing a multiplatform and open-source XR application. (c) GOSIM Workshop 2023 Sept 23-24 Grand Hyatt, Pudong, Shanghai https://workshop2023.gosim.org/ https://www.bilibili.com/video/BV18w411r7Nm/

1. 单击此处添加文本
GOSIM
Conference

2. Web browsing on
Extended Reality
Felipe Erias
2

3. Presentation
• Felipe Erias
Software engineer and interaction designer
felipeerias@igalia.com
• Igalia
Igalia is a Free Software consultancy specialised in
open source technologies on a wide set of platforms
https://igalia.com
3

4. Introduction
• Challenges and opportunities of Web browsing on Extended Reality
(
XR
)
,
drawing from Igalia’s experience creating the Wolvic browser
• Web browsing on XR enables users to navigate and interact with
traditional web-based content as well as immersive experiences
• Wolvic is an Open Source Web browser for XR devices which supports
immersive experiences
• Wolvic is the only multi-platform Web browser for XR
4

5. The traditional Web

6. Traditional Web browsing
• On desktop PCs and smartphones
• Windows and tabs, back/forward navigation
• Bookmarks, downloads, history
• Password management
• Support DRM media playback
• Extensions, private mode, etc.
• Some Web APIs need to be specifically implemented for XR
6

14. Input methods
• Typically, XR devices use handheld controllers and/or hand gestures
Other methods are supported by some devices
• But most of the Web is made for desktops and touchscreens
• This mismatch can be noticeable for content that is not designed for XR
• Websites may use input modes which can not be replicated easily with
VR controllers, like pinching and dragging, or keyboard shortcuts
• In Wolvic, we usually “pretend” to be a touchscreen or a desktop;
we are also working to integrate external keyboards and voice input
14

16. 16

17. Readability
• Reading documents is the most basic way to use the traditional Web
• On XR, the reading experience is limited by the resolution of the headset
• In Wolvic, we use a large window size with large text
• But there are several trade-offs with this solution
The screen can not be too large: it would cover the user’s field of vision
The text can not be too large either: the page would be harder to use
• Hopefully this limitation will get better as the hardware improves
17

19. Productivity
• The large virtual space provides great flexibility to the UI
• We can arrange content to leverage spatial thinking and memory
• We can experiment with richer ways to present information to the user
• UI elements can be positioned on the edge of the user's field of vision,
leveraging peripheral awareness
• XR has the potential to become a productive and flexible way to work
19

20. 20

21. 21

22. 22

26. Video and entertainment
• Usually provided as regular 2D video; also 3D stereoscopic video
• Popular streaming video platforms
YouTube, etc.
Platform must provide playback functionality via the Web (not only app)
DRM media content must be supported by the browser engine
Examples: Disney+, HBO Max, Prime, Xigua Video, Apple TV, U
-
NEXT…
26

27. 27
Disney+

28. Disney+: Spider-Man: into the…

29. YouTube: Graham Norton Show

30. YouTube: 3D Trailer
•
Avatar 2

31. The immersive Web

32. Immersive video
• Served over the Internet as a regular video media file
Can be distributed via YouTube, etc.
The video needs the right metadata so it can be displayed correctly
• The XR browser displays the video according to a specific geometry
Monoscopic: same image for both eyes
Stereoscopic: one image for each eye
Field of view: 180º, 360º
32

33. AirPano: Santorini, Greece

34. AirPano: Cancun, Mexico

35. 35
AirPano: Everest

36. Hellblade - delight-vr.com

37. Immersive experiences: WebXR
• WebXR is a collection of Web APIs that allow the development of
immersive XR experiences on the Web
https://immersiveweb.dev
https://immersive-web.github.io
https://www.w3.org/immersive-web
https://github.com/immersive-web
• In Wolvic, WebXR is mostly implemented by the Web engine, but
we still need to connect it to the graphical layers, input, etc.
37
WebXR

38. sprayspace.art

39. Brush Work

40. Silk Brush

41. Education
• Both immersive video and interactive content
• XR is the best way to experience immersive information
• The Web makes it very easy to distribute this content
No need to create a separate app, release it on each app store, etc.
• Even existing platforms like YouTube and WikiMedia can be used
41

42. YouTube: 360° Airline Pilot's View

43. anzcrewdg.360vr.world

44. nationalgallery.org.uk

45. magische-spiegelungen.de

46. magische-spiegelungen.de

47. accessmars.withgoogle.com

48. accessmars.withgoogle.com

49. accessmars.withgoogle.com

50. Gaming
• One of the most popular use cases for XR
• Limitations to deliver large assets
• Performance may be lower than in native applications
• Creative indie game developers
50

51. plockle.com

52. Boulderworld - heyvr.io

53. Moon Rider - moonrider.xyz

54. Moon Rider - moonrider.xyz

55. The Wolvic project

56. Wolvic: origins
• Wolvic started out as Firefox Reality
Initial release September 2018
Goal: bring Firefox to standalone VR headsets
Support the traditional Web plus immersive experiences
Part of a wider “mixed reality” initiative at Mozilla
• Handed over to Igalia in 2022
• https://wolvic.com
56

57. Components overview
• Custom 3D library (based on GLES2
)
• Web engine
(
Gecko, Chromium)
• Android UI components
• Mozilla mobile Android components (browser functionality)
• OpenXR to access device capabilities
• Some platform-specific libraries
57

58. Web engines
• Web engine: the component that interprets and renders Web content
• Gecko
Used by Mozilla Firefox
Provides a standalone Android library
• Chromium
Used by Google Chrome
Wolvic support currently under development
58

59. Multiplatform
• Wolvic uses OpenXR as abstraction layer to access most XR functionality
• But we still need to cover for devices with different capabilities
• We need to test specifically with each device (and work around bugs)
• We need to take device format into account:
VR headset, AR goggles, phone-tethered glasses…
• Different platforms provide different APIs and capabilities
For example, there isn’t an unified way to create a “desktop” app in XR
59

60. Developed in the open
• Mozilla Public License Version 2.0
• GitHub (code and issues): https://github.com/Igalia/wolvic
• Distribution
App stores: Huawei, Meta Quest, Pico XR
Side-loading packages: https://wolvic.com/dl/
• Funded through OpenCollective: https://opencollective.com/wolvic
60

61. github.com/Igalia/wolvic

62. wolvic.com/dl

63. opencollective.com/wolvic

64. Conclusions

65. Conclusions
• Web browsing on XR has great potential for a wide range of use cases,
from education and productivity to entertainment and gaming
• The Web lowers the cost of experimentation and distribution
• The Web can be a venue for innovation and creativity in the XR field
• Still open questions: UI design, input, interoperability…
• Wolvic is key as the open source and multi-platform Web browser for XR
65

66. Thank you!
felipeerias@igalia.com
https://wolvic.com
https://igalia.com

67. 单击此处添加文本

68. THANK YOU