UPDATED. The immersive industry needs an ethical framework to navigate moral dilemmas. This is my XR Ethical Manifesto presented at Greenlight's XR Strategy Conference on Friday, October 18, 2019 starting at 1:33p PT in San Francisco, CA.
27. ● Biometric data privacy implications
● XR modulates human perception
● Diversity of avatar representation
● Which biometric streams are personally
identifiable information?
● Cybersickness
● Correlation of behavior with genetics
● We should NOT have companies harvesting our
emotions
I. Self / Biometric Data / Identity Dilemmas
28. Biometric data is ephemeral &
context dependent.
Do real-time processing on it when
possible.
Don’t hoard it.
Imagine 10 years of biometric data
in the wrong hands.
An XR Ethics Manifesto - I
29. Offer a diversity of avatar
representations to increase the
fidelity of identity expression.
An XR Ethics Manifesto - I
30. Modulating perception & consciousness
is a super power.
Do no harm.
Ensure it’s consensual.
Use this ability wisely & responsibly.
An XR Ethics Manifesto - I
31. Conduct studies to assess the
long-term impacts of immersion.
An XR Ethics Manifesto - I
32. Nudging user behavior could have
unintended & negative long-term
impacts.
An XR Ethics Manifesto - I
33. Enhance the control & power of people
in an experience while respecting how
we’re interdependent.
We should be sovereign individuals
within the context of a collective.
An XR Ethics Manifesto - I
34.
35.
36. Yang Yin
Competition
Walled Garden
Curated Content
Vertical-Integration
Proprietary
App Store
Cooperation
Open Ecosystem
No Gatekeepers
Horizontal Platform
Shared Source Code
Open Access
38. ● Ethics of surveillance capitalism
● Corporations have a fiduciary responsibility to
shareholder profit, which can conflict with user
autonomy & privacy
● Do we have to pay for privacy?
● Can we own & sell our own data?
● Asymmetry of power in attention economy
● Can we share/loan virtual objects?
● Rights to ownership in virtual space
● Can differential privacy or homomorphic
encryption help privacy architectures?
II. Resources / Money / Values Dilemmas
40. You should own your data.
You should have a right to export,
sell, or permanently delete your data.
An XR Ethics Manifesto - II
41. Don’t aggregate financial transactions
across multiple contexts.
We should have the right to exchange
value without being tracked.
An XR Ethics Manifesto - II
42. Surveillance capitalism is
fundamentally unethical because it
changes the cultural & legal
definition of a “reasonable
expectation of privacy.”
This is due to the Third Party
Doctrine, which established that data
given to third parties is not private.
An XR Ethics Manifesto - II
43. Assume any de-identified data may have
a biometric fingerprint that could
unlock someone’s identity if
correlated with other Personally
Identifiable Information (PII),
observed behavior, or through machine
learning.
An XR Ethics Manifesto - II
44. Assume any data that’s tracked can be
leaked onto the Dark Web & into the
hands of tyrannical governments,
authoritarian adversaries, or
malicious hackers.
The best practice is not to record it
in the first place.
An XR Ethics Manifesto - II
45. We need new business models that don’t
create power asymmetries. Fiduciary
responsibility to profit can’t be
prioritized above the privacy &
autonomy of users.
An XR Ethics Manifesto - II
46. There is a digital divide preventing
equal access to these technologies. We
have a moral imperative to design
business models & political systems
that serve underrepresented minorities
across all classes.
An XR Ethics Manifesto - II
47. What we see as valuable changes over
time & algorithmic inference will
always be incomplete.
Be careful how these models create
assumptions about us that may be
completely wrong.
Offer ways to provide feedback or
mitigate the recording of information
when possible.
An XR Ethics Manifesto - II
56. ● Brain control interfaces will be able to read our
thoughts. Where’s that data go?
● Risks of negative transfer in education
● Design systems to mitigate hate speech
● Enable users to mute interactions
● Surveillance of private communication
● There’s also metadata surveillance tracking who
we’re talking to and when
● Implications of sharing biometric data?
● Consent for accessibility autotranscripts?
III. Early Education / Communication Dilemmas
58. We have a right to keep who we’re
communicating with & what we say
private.
We also need our privacy to be
balanced with security.
An XR Ethics Manifesto - III
59. Offer end-to-end encryption when
possible.
Don’t track who we’re talking to and
when we talk to them.
We need to balance freedom with
security, & continue to question which
takes priority based upon context.
An XR Ethics Manifesto - III
60. There is loss in all acts of
communication.
We must design systems that minimize
that loss & account for a spectrum of
abilities of communicative expression
& reception. It’s important that
everyone has a voice regardless of
ability.
An XR Ethics Manifesto - III
61. We’re moving from explicit data entry
to implicit data entry based on
embodied movements & biometric data.
We need standardized ways to port
individual profiles of trained input
gestures, movements, & voices to work
across different virtual worlds &
systems.
An XR Ethics Manifesto - III
62. Design systems that effectively handle
harassment and trolling.
Educate users of your prevention tools
as a part of your onboarding process.
Offer code of conduct orientations or
interactive training experiences to
help cultivate an inclusive culture.
An XR Ethics Manifesto - III
66. ● Who owns an AR space? Is it a new public
commons? Or determined by private property
ownership?
● Be aware of spatial doxxing threats
● What’s the ecological impact of tech?
● Threats of memory holing official history
● Tracking movements can reveal where you or loved
ones live
● Try to have a balance of private & public spaces in
your spaces.
IV. Home & Family / Private Property Dilemmas
67. It is everyone’s responsibility to
ensure technology is being produced &
materials sourced in an
ecologically-sustainable manner.
An XR Ethics Manifesto - IV
68. Virtual worlds can have public and
private spaces. Design private virtual
spaces to enable social experiences
that allow users more control,
autonomy, & identity than in public
spaces.
An XR Ethics Manifesto - IV
69. A home address is very sensitive to
doxxing. Educate users about the risks
of spatial doxxing in AR & how to
mitigate exposure of environmental
information that could used to
geo-locate them.
An XR Ethics Manifesto - IV
70. Immersive media can alter and enrich
our sense of the history and culture
of a place.
Never secretly alter historical
records in an attempt to misinform or
memory hole events.
An XR Ethics Manifesto - IV
71. Users must be educated on the risks of
allowing private spaces to be
volumetrically captured as it reveals
intimate personal & mental health
information. Otherwise they will
almost certainly be violating the
privacy of non-consenting friends &
family inadvertently.
An XR Ethics Manifesto - IV
78. ● Addiction & dopamine economy
● Conscious vs unconscious behaviors
● Consenting to violent content
● Unintended consequences of XR porn?
● Systems that limit creative expression
● Accidentally revealing sexual preferences through
eye tracking data
● Tracking of our entertainment options
● Mitigating sharing of explicit sexual content
V. Entertainment / Hobbies / Sex Dilemmas
80. Don’t use addictive gameplay mechanics
to create dependencies that disempower
users or degrade their quality of
life.
An XR Ethics Manifesto - V
81. Use content warnings, but be aware
some people may not know what they’re
consenting to. There is no way to take
an immersive experience back & there
may be unintended consequences to
experiencing violence in XR.*
*See Section VIII for more details.
An XR Ethics Manifesto - V
82. Emulate the view source principle of
the open web, which makes it
transparent & remixable.
Support WebXR & OpenXR to enable
freedom of creative expression &
innovation with open ecosystems rather
than walled gardens.
An XR Ethics Manifesto - V
83. Real-time biofeedback & implicit input
should not undermine users’ conscious
agency. Passive informed consent must
not be used to hijack autonomy against
the better interests of the user.
An XR Ethics Manifesto - V
84. Ensure XR porn is ethically produced
according to porn industry standards,
consent policies, & performer codes of
conduct.
Reduce shame around this topic by
engaging in open dialogue, & look for
ways to mitigate harm sustained from
porn addiction.
An XR Ethics Manifesto - V
85. Eye tracking & other biometric data
can be used to determine sexual
preferences, which can be
life-threatening for people in the
wrong country.
Design for the most marginalized
people in the most authoritarian
scenarios.
An XR Ethics Manifesto - V
86. Implement content moderation tools to
prevent the sharing of
sexually-explicit content on systems
frequented by minors.
An XR Ethics Manifesto - V
93. ● Do no harm
● Should you disclose medical conditions that you
discover in users?
● Flashing lights can trigger seizures
● Are people with specific mental health diagnoses
more susceptible to harm?
● Privacy for telemedicine
● Content trigger warnings & map cartography of
PTSD & trauma
● A rehabilitation path for banned users?
● Should HIPAA regulate biometric data?
VI. Medicine / Healing Dilemmas
94. All biometric data contains health &
medical information, & needs to be
protected as such.
The users’ data must be guarded
against malicious actors seeking to
capitalize by profiling through
behavioral inferences.
An XR Ethics Manifesto - VI
95. The storage of biometric data should
be regulated through something like
HIPAA or new regulatory frameworks
will need to be implemented.
An XR Ethics Manifesto - VI
96. Design technology to empower people to
monitor & assess health conditions
safely & effectively, thereby
providing more autonomy & control over
their own healing processes.
An XR Ethics Manifesto - VI
97. Design experiences for people on a
wide spectrum of physical conditions
and/or mental diversity.
An XR Ethics Manifesto - VI
102. ● New boundaries for intimate
relationships
● Designing for safety from harassment
● Mitigating deep fakes & forged identity
● Retributive vs Restorative Justice
● The downsides of empathy machine or
trauma tourism as tokenized spectacle.
● Dangers of anthropomophic AI influence
VII. Other / Partnerships Dilemmas
103. Provide tools for users to block and
mute people who harass them.
An XR Ethics Manifesto - VII
104. Implement methods for users to protect
the autonomy & safety of their
personal spaces (i.e. through personal
space bubbles or other innovations).
An XR Ethics Manifesto - VII
105. Improve upon self-sovereign identity
standards and invent new methods to
validate people’s identity and
mitigate harm caused by identity
theft, deep fakes, and bots.
An XR Ethics Manifesto - VII
106. Promote systems of restorative justice
within immersive technological
communities.
Cultivate practices of owning harm
done, authentic apologies, & paths
towards redemption.
An XR Ethics Manifesto - VII
111. ● Rights to identity after you die
● Implications of killing people in VR
● The right for you to be forgotten
● Using VR for torture
● Experiential warfare
● Long-term implications of exiling & permanently
banning people
● Sexual assault in XR
● Human rights violations
● Filtering violent or terrorist content
VIII. Death Dilemmas
112. We have the right to be forgotten.
Architect systems that allow people to
export their data or permanently
delete all traces of their digital
footprint.
An XR Ethics Manifesto - VIII
113. We must design trauma-aware interfaces
& annotate experiences with content
ratings to mitigate harm.
We have a right to produce content
exploring traumatic events, & also a
responsibility to do our best not to
cause harm.
An XR Ethics Manifesto - VIII
114. Consider the unknown ethical
implications of content you design.
Immersive violence may have unintended
consequences we cannot predict by
comparing it to 2D technologies.
Treat this new medium with the respect
unknowable & unpredictable technology
requires.
An XR Ethics Manifesto - VIII
115. Living & dying is a process we all go
through.
Honor how immersive experiences may
help with the grieving process, & be
encouraged to create new rituals
around death & loss.
An XR Ethics Manifesto - VIII
123. ● Many philosophical implications of XR
● What is truth? What is reality?
● Mitigating filter bubbles of reality
● Ethics of biohacking
● Need a comprehensive ethics framework
● Informed consent & progressive permissions
● What content is illegal?
● Implications of Third Party Doctrine
● Need new economic business models
● Future dream both protopias & cautionary tales
● Embrace paradox, plurality, process thinking
IX. Dilemmas: Philosophy / Higher Education / Law
124. Participate & initiate discussions
about ethical frameworks for XR.
Consider this XR Ethics Manifesto a
template & catalyst for the creation
of your own ethical frameworks.
An XR Ethics Manifesto - IX
125. Be aware of how the stories we tell &
worlds we build shape the future.
Dystopic narrative tropes can either
be cautionary tales or self-fulfilling
prophecies. XR is a powerful medium to
prototype worlds designed to cultivate
protopian cultures.
Create new worlds with deliberate
intention.
An XR Ethics Manifesto - IX
130. ● Mitigating governmental overreach, surveillance, &
tyrannical control
● XR used as governmental loyalty tests
● Conflicts of interest with industry & academia
● Using biometric data from XR for hiring
● Augmenting public spaces for good
● Implications of remote work
● Algorithmic transparency for AI
● Collective right to augment public space
X. Career / Government / Institutions Dilemmas
131. Don’t rely solely on biometric data
assessment for hiring.
Modelling is incomplete & can’t handle
the full complexity of a human
individual.
Don’t relegate human judgment to a
computer when judging humans.
An XR Ethics Manifesto - X
132. Enable algorithmic transparency so the
public can see how the architecture of
code is influencing their lives.
An XR Ethics Manifesto - X
133. Create public virtual spaces that are
open, vibrant, & encourage the freedom
of expression.
An XR Ethics Manifesto - X
134. Think critically about the
implications of technology across
different geopolitical contexts.
Authoritarian governments misuse &
abuse data for mass surveillance &
human rights violations.
Don’t enable turnkey tyranny within
your software architectures.
An XR Ethics Manifesto - X
135. WebXR content should be deliverable to
the web around the world independent
of country boundaries.
An XR Ethics Manifesto - X
139. ● Danger of explicit & implicit social scores
● How to support the cultivation of communities
● Code of conducts & enforcement
● Violating normative standards (playing Pokémon
Go at The Holocaust Museum)
● Freedom of assembly & association implications
● Principles of diversity & inclusion
● Preventing algorithmic bias
● Biometrics data radiated to community
XI. Friends / Community / Cultural Dilemmas
141. Cultivate cultures, don’t engineer
them.
Driving collective behaviors requires
many ethical considerations, don’t
take shortcuts.
When it doubt, choose to benefit the
many over the few.
An XR Ethics Manifesto - XI
142. Diversity & inclusion is a
foundational principle, & should be
encouraged at all levels because it
benefits the collective.
An XR Ethics Manifesto - XI
143. Promote diversity of input & feedback
as a way to mitigate algorithmic bias.
Raise the accuracy of your algorithms
by including data sets & stakeholders
from diverse backgrounds.
An XR Ethics Manifesto - XI
144. Have a code of conduct for your
community, experience, or product.
Behavior is modelled both from the top
down as well as the bottom up.
Encourage users to cultivate the
cultures they want.
An XR Ethics Manifesto - XI
145. WebXR content should work across all
of the major XR platforms & devices.
An XR Ethics Manifesto - XI
150. ● Inclusive design & accessibility
● Is it possible to be truly anonymous?
● Dark Spatial Web
● Biometric polygraph
● Re-identifying de-identified PII data
● Bad inferences from incomplete context
● Utility vs Downsides of exile or banning
XII. Hidden / Exiled / Accessibility Dilemmas
151. Make XR content available to all
people regardless of physical ability
or fidelity of input.
Making content accessible improves the
experience for everyone.
Solicit feedback from people with
disabilities for how to make your
experience more inclusive.
An XR Ethics Manifesto - XII
152. Enable consumers of WebXR content to
modify how content is rendered in
similar way to how they can with the
2D web.
An XR Ethics Manifesto - XII
154. ������
��
Domains of
Human Experience
(Contexts)��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Self
Biometric Data
Identity
Resources
Money
Values Early Education
Communication
Home & Family
Private Property
Entertainment
Hobbies
Sex
Medical
Heath
Other
Partnerships
Law
Death
Collective Resources
Philosophy
Higher Education
Career
Government
Institutions
Friends
Community
Collective Culture
Hidden
Exiled
Accessibility
155.
156. This is the start of the
conversation.
We need everyone’s
insights & input.
158. Slides from My “Ethical
& Moral Dilemmas of
Mixed Reality Talk at
Augmented World Expo
159. ������
��
Photo by
Domains of
Human Experience
(Contexts)��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Self
Biometric Data
Identity
Resources
Money
Values Early Education
Communication
Home & Family
Private Property
Entertainment
Hobbies
Sex
Medical
Heath
Other
Partnerships
Law
Death
Collective Resources
Philosophy
Higher Education
Career
Government
Institutions
Friends
Community
Collective Culture
Hidden
Exiled
Accessibility
160. ������
��
Domains of
Human Experience
(Contexts)��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Self
Biometric Data
Identity
Resources
Money
Values Early Education
Communication
Home & Family
Private Property
Entertainment
Hobbies
Sex
Medical
Heath
Other
Partnerships
Law
Death
Collective Resources
Philosophy
Higher Education
Career
Government
Institutions
Friends
Community
Collective Culture
Hidden
Exiled
Accessibility
161. ������
��
Photo by
Self
Biometric Data
Identity��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Biometric fingerprint?
Modulate perception
Diversity of avatar options
Self-sovereign identity
Movements as PII?
Who owns your data?
Sell access to your data?
Privacy as a luxury good
BCI reading thoughts Correlate genetics to
behavior
Biofeedback entertainment
Unconscious vs Conscious
Agency
Ethics of detecting & reporting
medical conditions
Informed Consent
Progressive Permissions
Deep Fakes of Your Identity
Autonomy of data sovereignty
vs Utilitarian public good
Ethics of biohacking
Biometrics used in interviews
Government loyalty tests
Sharing biometrics with friends
Implicit bias
Biometrics as polygraph
Anonymity vs biometric security
False positives from missing context
Can biometrics be de-identified?
�� ��
Mortgaging your privacy
Others profiting on your data
Harvesting your emotions
162. ������
��
Photo by
Communication
Early Education��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Eye tracking & emotions
Physiological development of
children
What age is safe? How long?
Free Educational Resources
End-to-end encryption
Peer-to-peer connections
Being able to work from
home`
People sharing inappropriate
sexual content
Privacy of Telemedicine consults
1st Amendment
Free Speech
Hate Speech
Education for EveryoneRemote Telepresence
Code of Conduct
Banning people
Anonymous communication
�� ��
Negative Transference
163. ������
��
Photo by
Home
Family
Private Property��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Scan your home & the objects
in it
Burglers scoping out your
valuables
Protecting home
address
Giving consent for others
to use your property for
augmentation
4th Amendment and the
“Reasonable expectation
of privacy”
Ecological sustainability of XRCollective right to augment
Public spaces
Pokemon Go at Holocaust
Museum
Spatial doxxing
�� ��
Rehabilitation & exercise at home
165. ������
��
Photo by
Medical
Health��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Cybersickness
Individualized medicine
Become your own healer
Secure communications Own your own medical
data
Ethics of detecting & reporting
medical conditions
Should HIPAA govern biometric
data?
New models of science from
big data to do individual
assessments and
personalized medicine
Public research benefit from
sharing biometric data
Community healing rituals
Privacy in telemedicine
Homomorphic encryption
Differential Privacy
�� ��
166. ������
��
Photo by
Other
Partnerships
Law��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Violation of personal space
Avatar discrimination
Sharing virtual resources
with your friends Mute others
Hate speech
Having private spaces
to gather in
Augmenting other people
New boundaries for intimacy
Anthropomorphizing AI agents
Empathy
Accountability
Comprehensive frameworks
For privacy
XR Ethical Frameworks
Restorative justice
Moderators enforce the
Rules & code of conduct
Implicit trust & reputation scores
Cultural norms
Block harassment & trolls
Possible to be an anonymous user?
��♀ ��
Illegal content
Sexual assault in XR
Blocking others
167. ������
��
Photo by
Death
Collective Resources��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Right to have your data deleted
Ephemerality of biometric data
Rights to your likeness after
you die
Don’t record metadata
of communications
Killing people in virtual worlds
Rehabilitation of blocked users
Implications of permanent
blocking?
Taxes for public services?
Taboos around death
⚰ ��
Virtual funeral rituals
168. ������
��
Photo by
Philosophy
Higher Education��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Siloed academic disciplines
Conflicts of interest of
academic collaboration
with industry & IP
Disclose what’s “real”
and what’s “not real”
Filter bubbles of reality
Is direct experience basis of
“reality” for harassment
What is Reality?
Definition of consciousness
Ethical frameworks
World build “Protopias”
More interdisciplinary
collaborations between
academia and industry
Diversity of data for training AI
�� ��
Technical debt of machine
learning
Visualizing dystopian futures
169. ������
��
Photo by
Career
Government
Institutions��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Using biometric data as a
Polygraph test
Legally-mandated,
profit-driven maximization
of shareholder value
Memory holing history
Augment historical
monuments
Third party doctrine
NDAs
Experiential Warfare
GDPR for USA?
TransparencyGovernment surveillance
Open source algorithms
Social credit scores impact
Access to physical services
Undermining democracy via
leaked psychographic profiles
�� ��
VR Torture
Government getting secret
access to our behavioral &
biometric data streams
Government
getting access to
what we say online
171. ������
��
Photo by
Hidden
Exiled
Accessibility��
�� 🏠
👪
����🏽
��
��
��
��♀
⚰��������������
��
��
Design for non-abled bodies
Reading thoughts
Eye tracking accessibility apps
Paralyzed
Blind
Deaf
What we value = what we
look at Autotranscripts for deaf Living permanently in
Virtual worlds
Rehabilitation of trolls
Restorative justice for banned
Illegal content in XR (child porn?)
Map the unconscious psyche
Unknown long-term effects
Dark spatial web
�� ��
Harvesting your emotions