AR and VR for Good

AR and VR for Good
Research Workshop Seminar
Centre for Research-Creation in Digital Media
Research: zisiangsee@sunway.edu.my
FB: www.facebook.com/zisiang
Research track: www.zisiangsee/research
Presentation: Zi Siang See
7 June 2017
Research &
collaboration

Is user experience important to you?

Researcher: Zi Siang See
Creative Industries and IT
Business and Corporate Branding
Industrial Involvement since 2003/04

Research and Specialization
Professional Interest
• Creative Advertising
• Computational Photography
• Interactive Multimedia
• Virtual Reality 360 (VR360)
• Augmented Reality
• Imography Research
• Human Computer Interaction

Work: McDonald’s Malaysia
Creative: NagaDDB
Operation: Quality Ingredient Story
Creative Imaging: Reina Imaging
Corporate Branding

Digital Imaging (DI) for Creative Advertising Visuals

Digital Imaging (DI) for Advertising Visuals

Interactive Online Contenthttp://www.zisiangsee.com/vtour/vtour_hdr_sphericalpanorama/index.html

UC Berkeley
Collaboration

Founding Projects
The Project

Figure: VSMM2016 also features the system by Prof Sarah Kenderdine

Figure: Emphatic Computing Lab, University of South Australia

Introduction
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve

Introduction
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
• Touch
• Scent
• Taste
• Sight
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve
Development /Trend
• Augmented and Virtual Reality
• Human Computer Interaction

Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard.

Desk /Table Pocket /Palm /Hand Head /Eye Ubiquitous
Major changes in computing platforms

Virtual Reality
Figure: NASA, 1988 Virtual Environment System

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=NAuytnYU6JQ

Figure: Super cockpit project, Tom Furness.

Near-eye-display and fully augmented interactive display experience
How far can we go?

Augmented Reality

Augmented Reality: Glass
Exploration in modern wearable interface
• Fashionable, UX practicality?

Google Glass
• Google Glass, developer version, 2013
• Can be configured on user’s customized or personal spectacles.

Recon Jet
• Sporty design
• Cost “friendly”
• Smartphone compatible
• External Connectivity

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=BhBtuNe_8is

• Displays, tracers, and AR systems in general need to become more accurate, lighter,
cheaper, and less power consuming (Azuma, 2001).
Figure: wearable computing, Steve Mann.

ideas worth spreading:
How can we develop AR and VR
content or systems for good
purposes?

Augmented, Virtual, Mixed Reality
• Effects of Sharing Physiological States of Players in a Collaborative Virtual Reality
Gameplay. (Dey et al., 2017)
• Digital Taste Interface project, Electronic taste stimulation (Ranasinghe et al., 2011)
• Billinghurst, M. (2011). The Future of Augmented Reality in Our Everyday Life.

AR and VR Pillar: User Experience
AR and VR Pillar Project

Google Cardboard

Google Cardboard
Programmable NFC
Tag
Low cost resin
/plastic lens
Mobile device as
computing module
Endless configurable size and
shapes (and materials)
Magnetometer

Google Cardboard v2
Presses the
screen

What do I see?
http://www.zisiangsee.com/vtour/vtour_hdr_sphericalpanorama/index.html Presentation: Zi Siang See

Researcher: Zi Siang Seehttps://www.youtube.com/watch?v=hjKd24UCPYY

Samsung Gear VR
• Samsung introduced Gear VR in October 2014, co-developed with Oculus. Gear VR can
house specific mobile device for providing AR & VR user experience (Samsung, 2014)
• Control interface included

Samsung Gear VR
• VR /AR for end users
• Social Media?
• UX prototyping
Figure: consumer product of HMD, suitable for AR and VR content

Gear 360

Gear 360
Lens 1 far from Lens 2
Lens 2 far from Lens 1
Nodal point is still not
centered, this can cause
serious parallax error
Figure: commercially available consumer 360 camera

AR and VR for Public Events
• User Experience is the focus
• Distributable, medium of communications, believable
Figure: iLabs launching, 05 April 2017

Oculus Rift
Figure: Oculus, by Facebook

Android VR
Figure: Android VR collaborating with other developers
Google has announced eight hardware partners that will make Daydream-ready phones,
including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel.

Android VR
Google has announced eight hardware partners that will make Daydream-ready phones,
including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel.

Apple + Metaio?
Metaio was one of the most technologically-ambitious AR organization, acquired by Apple in 2015

Apple’s Recent Initiatives
SteamVR Coming To Mac (unveiled in WWDC 2017), AR Kit will be available too.

Steam VR
Presentation: Zi Siang SeeVinay Narayan - HTC Vive (https://visionsummit2017.com)

54% Want To Use VR/AR In Physical Stores
• 85% -- Travel experiences
• 84% -- Museums and galleries
• 84% -- Education, training
• 83% -- For designing something
• 79% -- Live entertainment
• 77% -- Exercise or sport
• 75% -- Navigation
• 74% -- Retail and shopping
• 70% -- Business
• 69% -- Meditation
• 63% -- Fully immersive gaming
• 53% -- Gaming through mobile
• Comprised interviews of 16,000 consumers.
• Consumer's perspective: the main reason for not having tried VR or AR is the lack of
opportunity, based on a new study.
https://www.mediapost.com/publications/article/301863/54-want-to-use-vrar-in-physical-stores.html

Media Companies
Corporate media investment into AR/VR startup - infographic.

User Experience

User Experience
Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=sfowhrpS-6M

Definitions
Virtual Reality
• Virtual Reality (VR) which can be referred to as immersive multimedia or computer-
simulated life, replicates an environment that simulates physical presence in places in the
real world or imagined worlds and lets the user interact in that world. Virtual reality
artificially creates sensory experiences, which can even include sight, hearing, touch, and
smell.
Augmented Reality
• Augmented Reality (AR) mixes a live real-world view with virtual interactive content on a
mobile or wearable device. One of the key enablers for this is tracking technology, such
as computer vision techniques for tracking off pre-defined markers or markerless images.

Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.

AR + VR + MR = XR?

Gartner’s Hype Cycle

Platforms
VR AR
Sourced and reinterpreted from
http://www.augment.com/blog/which-headset-is-right-for-you/

AR & VR for Good

Through Heart and Eyes:
Sharing what you feel and
interacting with what you see
(Thammathip Piumsomboon, 2017)

Figure: Quiver, the 3D Augmented Reality coloring book technology, originally called colAR, was developed at the HIT Lab NZ.

AR for medical purposes
Figure: The first head-mounted display (HMD)-based AR
system was described by Sutherland in 1968
Figure: Invention timeline of selected imaging and AR
technology © 1968 IEEE
Tobias Sielhorst, Marco Feuerstein, and Nassir Navab (2008) Advanced Medical Displays: A Literature Review of Augmented Reality.
Journal of Display Technology, vol. 4, no. 4, December 2008.

How One Dad Is Using VR To Help His Daughter
https://uploadvr.com/vr-good-one-dad-williams-syndrome
Figure: Craig and Imogen Chaytor's Battle With Williams Syndrome

Hair Raising Message

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=064ipuBiWDg

Practical consideration
• Ubiquitous computing – the installation of hardware are invisible to users
• Focus on user experience
• Has strong storyline
• Brings social good and awareness

Penguin Navi

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=IK4-zPD_25U

• Mix computing and entertainment
• High level of usability design
• Focus on bringing the intended outcome – going to the aquarium
• Make use of existing mobile devices and capability

Empathy Glasses
Katsutoshi Masai, Kai Kunze, Maki sugimoto, Mark Billinghurst (2016) Empathy Glasses.
Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=CdgWVDbMwp4

• Emphatic computing
• Shared experience, emotion, same viewpoint
• Real-time
• “Put myself in their shoes”
• Achieve and complete tasks together

Huggy Pajama
James Keng Soon Teh, Adrian David Cheok, Roshan L. Peiris, Yongsoon Choi, Vuong Thuong, Sha Lai (2008) Huggy Pajama: a mobile parent and child hugging communication system.
Proceedings of the 7th international conference on Interaction design and children.
Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=hQ6usrx-GPM

Mobile Audible AR
Zi Siang See, Mark Billinghurst, Vimalan Rengganaten, Shelly Soo (2016) Medical Learning Murmurs Simulation with Mobile Audible Augmented Reality.
Siggraph Asia 2016 Mobile Graphics and Interactive Application

Mobile Audible AR
https://www.youtube.com/watch?v=VGsaTDKC2qY

Virtual Reality in Medicine Leadning
University of Basel, Switzerland
https://www.unibas.ch/en/News-Events/News/Uni-Research/Virtual-Reality-in-Medicine.html

Presentation: Zi Siang Seehttps://youtu.be/AttXbcLUyR0

Virtual Monitors, Navigation
Figure: over 10,000 surgeries have been successfully completed using Scopis technology and the Navigation Platform has a lot of potential for use beyond spinal surgery.
https://uploadvr.com/berlin-company-using-hololens-improve-surgical-procedures/
Presentation: Zi Siang Seehttps://youtu.be/xvbWE4OsKxY?list=PLCHEesPXRgF74fGSPd6SK-mda35E32up-

Work Smarter, Improve Safety
Figure: Elevator technicians are using AR to improve safety and workflow
https://uploadvr.com/microsoft-hololens-elevator-technicians/

VRPhysio

VRPhysio
• Customizable
• Clinic can track and help patients remotely
• Analytics

Presentation: Zi Siang Seehttps://www.youtube.com/watch?v=WmSyWb6djDU

Research Possibilities
Exploration
• Method and apparatus for augmented reality
• Method and apparatus for virtual reality
• Ubiquitous computing
• Obstacles and issues
• Technology management
• Education technology
• Engineering and concept design
• User interface (UI) design
• User experience (UX)

Can we use VR/AR technologies to help more people?
Effort
• Empathic computing
• Experience synchronization
• Improve well-being
• Medical-based benefits
• Archeology, cultural heritage: see the whole story in 360 (digital preservation)
• Sharing community and social stories
• Combine interest of different professional disciplines

Some on-going Projects

“Creativity is not an option, it’s an absolute necessity”
Sir Ken Robinson 2014

Figure: European researchers used 3D-modeling and Instagram to recreate Mosul Museum destroyed in civil war.

Virtual Reality 360
Researcher: Zi Siang Seehttp://www.zisiangsee.com/vtour/vtour_hdr_sphericalpanorama/index.html

VR brings ninth century Viking invaders' camp to life (YorkShire Museum, 2017) - staged in partnership with the British Museum, new research by the universities of York and Sheffield.
https://www.theguardian.com/artanddesign/2017/may/17/virtual-reality-brings-ninth-century-viking-invaders-camp-to-life

Figure: VSMM2016 featured the system by Prof Sarah Kenderdine

Figure: VSMM2016 featured the system by Prof Sarah Kenderdine
Pureland: Inside the Mogao Grottoes at Dunhuang

AR and VR Pillar (2017)
Director: Prof Harold Thwaites
AR and VR: Zi Siang See
Audio Visual: Delas Santano
3D: Human Esmaelli
AR and VR Pillar

User Experience Study
• User experience found VR360 has highest rank in ease-of-use and usefulness
• Study also found AR with 3D scored second in preferences (but highest rank in mentally
and physically challenging)
Ease-of-use
Usefulness

Possible improvements
• Users can choose between online or offline application.
• Exhibitor-provided devices can be preloaded with content.
• Miniature AR and VR pillar can be purchased or given away as souvenir which comes with
simplified interactive features.

Park Bench Sojourn
Researcher: Zi Siang Seehttp://www.parkbenchsojourn.org/
(Matthew Sansom)

Building Information Modeling VR
On-going Project Development
• FARO 3D Scanning (University of Reading)
• VR with freedom of room-scale locomotion (Sunway University)
Research
collaboration

3D-fying our world: Archiving
3D Scanning using FARO
• Trained users, costly, precision CAD content

VR for Disaster Relief
• Asia-Pacific SpatioTemporal Institute (Taiwan)
• Location-based VR visualization
Research
collaboration
Semi-automated
post-processing
Acquisition and
Technical Setup
User experience
/Distribution
Disaster sites:
- scenario?
- consideration?
UC Berkeley
ADVANCED DIGITAL IMAGING FOR
PRESERVATION, CONSERVATION
AND EXHIBITION DESIGN
SCHEDULE. 11th
– 13th
SEPTEMBER 2013
WEDNESDAY 11th
SEPTEMBER
9:30 – 9:40 Welcome to workshop DR. SARAH
KENDERDINE(SK)
9:40 – 10:00 Welcome to HKSTP, ALiVE& LUXLAB
10:00 – 10:15 Outline of workshop
10:15 – 11:15 Demonstration of optical scanning DR. JAYARRETOQUE
(JA)
Ms. Mie KADO,
Mr Kouji OKUMURA
11:15 – 12:30 Introduction to immersive museums SK
12:30 – 1:30 Highlights fromoptical scanning projects DR. JAYARRETOQUE
1:30 – 2:30 Lunch
2:30 – 3:30 Tour of ALiVEexhibitions DR. KENDERDINE
3:30 – 5.00 Highlights fromoptical scanning projects PROF. ARI IDE-
EKTEBASSI (AIE)
5:00 – 5:30 Q& A TEAM
5:30 – 6:30 Transfer to Diamond Hill by bus OPTIONAL(please
register with Kyle Chung
for catering purposes
before 1st September)
6:30 – 7:30 Tour of NamLian Gardens < nlgarden.org>
7:30 – 9:00 Vegetarian banquet at the Chi Lin Restaurant at
NLG(HK$450 & *students HK$225 per head)
THURSDAY 12th
SEPTEMBER
9:30 – 10:30 Ultra high resolution and high colour definition AIE
10:30 – 12:00 Analytical Imaging I: Introduction to Analytical
Imaging
JA
12:00 – 12:30 Introduction to Hong Kong Maritime Museum
project
MR. ROBERTTRIO
12:30 – 1:00 Q& A TEAM
1:00 – 2:00 Lunch
2:00 – 4:30 Scanning and training JA
4:30 – 6:00 Tour of Creative Media Centre exhibitions
including the Pure Land projects (optional).
Proceed to venue by taxi.
SK
Asia-Paciﬁc SpatioTemporal Institute
UC Berkeley
INTERNATIONAL WORKSHOP ON
ADVANCED DIGITAL IMAGING FOR
PRESERVATION, CONSERVATION
AND EXHIBITION DESIGN
SCHEDULE. 11th
– 13th
SEPTEMBER 2013
WEDNESDAY 11th
SEPTEMBER
KENDERDINE(SK)
(JA)
Ms. Mie KADO,
Mr Kouji OKUMURA
1:30 – 2:30 Lunch
EKTEBASSI (AIE)
5:00 – 5:30 Q& A TEAM
THURSDAY 12th
SEPTEMBER
Imaging
JA
project
MR. ROBERTTRIO
12:30 – 1:00 Q& A TEAM
1:00 – 2:00 Lunch
SK

Virtual Reality Cultural Heritage
• Electronic Cultural Atlas Initiative (ECAI) & CRCDM
• Bujang Valley - digital preservation research-creation
Research
collaboration
UC Berkeley
KENDERDINE(SK)
(JA)
Ms. Mie KADO,
Mr Kouji OKUMURA
1:30 – 2:30 Lunch
EKTEBASSI (AIE)
5:00 – 5:30 Q& A TEAM
THURSDAY 12th
SEPTEMBER
Imaging
JA
project
MR. ROBERTTRIO
12:30 – 1:00 Q& A TEAM
1:00 – 2:00 Lunch
SK

Museum VR: ECAI
• Electronic Cultural Atlas Initiative
• User experience and digital preservation of cultural artifacts

Museum VR: The Tomb of Sultan
• 3D Scanned /Photogrammetry of artifacts
• User experience –revisiting an ancient site where the king rested.

Museum VR: The Tomb of Sultan – Interaction?
• Can we see the hands? And can we gain some control? Physics?

Museum VR: The Tomb of Sultan – Interaction?
• Hands VR assisted by Paul Tschirgi (2017), allowed to be customizable.

MOOC:
Virtual Reality, Augmented Reality:
The Promise of Sci-Fi?
Research &
collaboration
Massive Open Online Course: AR and VR
Preparation: Zi Siang See
To be available soon
Center for Research-Creation in Digital Media

Can we develop systems that
allow us to share what we are
seeing, hearing and feeling
with others?
(Empathic Computing, Mark Billinghurst)

Thank you
Research Workshop Seminar
Centre for Research-Creation in Digital Media
Research: zisiangsee@sunway.edu.my
FB: www.facebook.com/zisiang
Research track: www.zisiangsee/research
7 June 2017

AR and VR for Good

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (19)

Semelhante a AR and VR for Good

Semelhante a AR and VR for Good (20)

Último

Último (20)

AR and VR for Good