Designing more ethical and unbiased experiences

Designing more ethical
and unbiased experiences
This work is licensed under a Creative Commons
Attribution-NonCommercial 4.0 International License
except where noted otherwise.
Karen Bachmann Carol Smith
@KarenBachmann @Carologic

#AbstractionsCon @KarenBachmann @Carologic
What do we need most
Awareness, ourselves and our colleagues
Exposure to other perspectives
Connection with our own humanity

“
“AND ETHICS IS SUCH A NEGATIVE SUBJECT.”
Many people believe that embracing ethics would limit
their options, their opportunities, their very ability to
succeed in business.
John C. Maxwell, Ethics 101
(CEO considering ethics as a topic for a sales meeting)

Empathy

Empathy vs. Ethics?

A progression for doing good
LawsNorms/CodesEthics
Values/morals
Empathy
Individual Society

Ethics

Ethics
Based on well-founded standards of right and wrong
Standard of expected behavior that guides the correct course of action
What is Ethics? By Manuel Velasquez, Claire Andre, Thomas Shanks, S.J., and Michael J.
Meyer. Markkula Center for Applied Ethics
https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/what-is-ethics/

What is Ethics? By Manuel Velasquez, Claire Andre, Thomas Shanks, S.J., and Michael J.
Meyer. Markkula Center for Applied Ethics
https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/what-is-ethics/
Ethics in terms of…
Rights
Obligations
Benefits to society
Fairness
Specific virtues

Does the Trolley Problem Have a Problem? What if your answer to an absurd hypothetical question
had no bearing on how you behaved in real life? By Daniel Engber. Slate.com. June 18, 2018.
https://slate.com/technology/2018/06/psychologys-trolley-problem-might-have-a-problem.html
Not the Trolley Problem

No absolute right or wrong or even
catastrophic failure – but a cost to
be weighed
Taking a broader perspective to
include extended communities, the
environment, and other
“stakeholders”
Resource: HBR articles that explore
real, nuanced situations
MIT Moral Machine Project
Complex problems need sophisticated tools

● Pace of change
○ Avoid creating fear, uncertainty, doubt, anger, and unrest
● Counter perceptions in the industry
○ Unethical behavior, arrogance, and lack of empathy
● Consider ramifications
○ Don’t jump into the new and the cool too quickly
● Understand nuances of behavior and habits
Ethics is needed in tech

Addressing ethics is for the entire industry
Product owners
QA testers
Developers
BAs
Data scientists
Project managers
UX researchers
Designers

● Consider human need and benefit fully in our work
○ Consider full human context over time and social ecosystem
○ Mitigate the thrill of the “new”
● Counter
○ Emphasis of technology over humanity
○ Primacy of "data" over people
● Raise visibility of actual people, rather than getting lost in data
What can tech professionals do?
Ethics offer a pragmatic path to supporting these goals and the conversations around them

“
ETHICS DOESN’T HAVE TO BE COMPLICATED
When you know better, you do better.
Betty Easterly
It’s about being aware and informed

Bias

Content
Design
Data
Solutions
...Are Biased

Due to…
Social class
Resource availability
Education
Race, gender, sexuality
Culture, theology, tradition
More…

So many biases!

The right humans
are required to
reduce bias

Bias?

Need to talk
about biases
in non-threatening,
productive ways

Bias: A Mental Shortcut
Part of being human
Kahneman’s System 1
Not inherently bad, may be misapplied
Implicit = invisible
Not necessarily in sync with
our conscious beliefs
Can be managed and changed

Start with building
awareness and
knowing ourselves

Know thyself
Start with understanding your own bias
Don’t fall into the illusion of complete objectivity
Don’t be afraid to acknowledge bias

Scenarios

Assumption That Math Reduces Bias
Amazon’s computer models trained to vet
applicants
- observed patterns in resumes
- most came from men
Taught itself that male candidates preferable
Penalized resumes that included the word
“women’s,” as in “women’s chess club
captain”
Downgraded graduates of two all-women’s
colleges

Embedding Existing Bad Behaviors
Both online and face-to-face mortgage
lenders charge higher interest rates to black
and Latino borrowers
Costing homebuyers up to half a billion USD
more in interest every year than white
borrowers with comparable credit scores
“Even if the people writing the algorithms
intend to create a fair system, their
programming is having a disparate impact on
minority borrowers — in other words,
discriminating...”

Dehumanizing thru Math

Not Anticipating Bad Behavior

Lacking Plan for Mitigation

Be Intentional
Make it your
business to keep
people safe

“Ensure humans can unplug the machines”
– Grady Booch, TEDTalk
Grady Booch, Scientist, philosopher, IBM’er
https://www.ted.com/talks/grady_booch_don_t_fear_superintelligence

Benefits

vs.
Making ethics a success goal

Easy to find examples of unethical behavior

● Focus on value
● Measure success gained, not
penalties avoided
● Recognize opportunity and
growth, not constraint and
stagnation
● Avoid negativity fatigue caused
by only focusing on shocking
failures
The “carrot” more compelling than “stick”
● Think of ethics on a larger
scale
● More than "treat your
participants well"
● Avoid focusing on getting
“caught” because it
encourages only the
minimum

● It reduces business liability.
● It helps employees make good decisions.
● It assures high-quality customer service.
● It prevents costly administrative errors and rework.
● It consistently grows the bottom line.
Support: National Ethics Association

Companies with good ethical policies earn:
● Marketing advantages over their competitors. Customers readily invest in the companies through
shares and also want to establish long lasting business relations with the company.
● The performance of employees improves with good ethical policies present in a company. Morale
is high and employees feel obligated to put in their all to continue to make it a success.
● Reputation management: A bad reputation is created by unethical behaviour which will eventually
lead to a scandal. A scandal will result in falling stock prices, anxiety, and low morale among
employees as well as government and public scrutiny and inquests.
● Legal and financial incentives: Companies known for their high ethical standards and education of
employees on ethical policies are provided with strong legal and financial incentives by regulatory
bodies.
Support: SixSigma Online

“
A better motivation for inspiring ethical behavior is the benefits it provides, both personally and
financially. Besides feeling good about doing the right thing, principled business conduct can also be
profitable. Ethical behavior is good business.
Four key benefits:
● Easier accounting – real numbers and honest account is easier than maintaining falsified records
● Better branding – creates trust with employees, suppliers and customers
● Improved bottom line – retain customers, earn high ratings, and gain referrals
● Better health – Reduce stress and feel better about your business
Support: Steve Parrish in Forbes
https://www.forbes.com/sites/steveparrish/2016/02/04/the-profit-potential-in-running-an-ethical-
business/#654f4a9e7687

“
QUOTING JAMES BURKE
If you invested $30,000 in a composite of the Dow Jones
thirty years ago, it would be worth $134,000 today. If you
had put that $30,000 into these [socially and ethically
responsible] firms - $2,000 into each of the fifteen [in the
study] – it would now be worth over $1 million.
John C. Maxwell, Ethics 101
Chairman of Johnson and Johnson

Resources and Tools

“
Tools for ethical design
If it's not ethical, it cannot be beautiful.
Yves Behar
How can we apply our understanding of ethics?

● A commitment to doing the right thing
● Clear standards
● Organizational values put into action (by leaders and employees)
● Accountability
● Open communications up, across and down the organization
Attributes of ethical organization cultures
Steve Priest in “Ethics Exchange: Culture (Part Two): Characteristics
of Ethical Organizational Cultures”

● Understand your values before a crisis
● Establish what it means to do good, not merely avoid doing harm
● Consider context as well as tactics
● Choose your work as carefully as possible
Know your ethical boundaries

Know your limitations: Karen’s IAT results on disability

“
ETHICAL ALIGNMENT - CODES?
Be conscious and aware of what you’re endorsing with
your time, not only for the welfare of others but for your
own sense of self. The work you choose to take on
defines you.
Stephen Anderson
Making sure we are in the right place

● Support and bolster your colleagues’ ethical practice
○ Share case studies and techniques
○ Adhere to and promote ethical practices
○ Amplify successes
● Protect integrity of industry best practices by setting a standard
of ethical design practices as the norm
Promote professional boundaries

“
CONSIDER ETHICS WHEN DESIGNING NEW TECHNOLOGIES
What is needed is strong, anticipatory guidance by those
who intersect the technology, health and ethics worlds to
determine how we develop and deploy technologies that
deliver the greatest societal benefits.
Christie and Yach, TechCrunch
Reacting to Facebook’s dilemma with fake news

Code of ethics
WIKIPEDIA “PROFESSIONAL ETHICS”
Most professionals have internally
enforced codes of practice that
members of the profession must
follow to prevent exploitation of the
client and to preserve the integrity of
the profession.

Code of Conduct / Ethics
What do you value?
What lines won’t you cross?
Track progress?

● Technical education including ethics in the curriculum
● Ethical design methodologies, heuristics, guidelines, practices
● Increasing awareness and ongoing discussions
Do we have the tools?

● New thinking: Cradle-to-cradle or closed loop thinking
● Biomimicry, nature as inspiration
● Focus on human and ecosystem benefit and betterment
● Focus on design over time and evolving scenarios
Rethinkinghow we do our work

● Montreal Declaration for Responsible AI
● Ethical OS
● Comparison of various tools (Principled AI: Mapping Consensus
and Divergence in Ethical and Rights-Based Approaches)
● Organizational questions (see appendix)
Ethical Tools

Most tools start with a set of questions
● Who?
○ Directly affected
○ Indirectly affected
● What is one benefit gained by someone
in each group?
○ Initially
○ After using the product over time
● What is one harm those same people
might experience?
○ Initially
○ After using the product over time
● Can the harm be mitigated? Does it
require a trade-off against a benefit?

Introspective Detection Tools
● Changing decision making
approach
○ Pre-mortems
○ Worst case scenario
○ Multiple estimations -> designs
○ Seek outside view
● Implicit Association Test
● Flip it to test it
● Invite examination and input
from people who are not like “us”
● Appoint a Devil’s Diversity
Advocate
● Slow down and reflect

The challenge
is less the lack
of tools, but the will
to use them

Continue the conversation...
LinkedIn:
CarolJSmith
KarenBachmann-UXResearcher
Twitter:
@Carologic
@karenbachmann
Slideshare:
carologic
karenbachmann
Toward ethical, transparent and fair AI/ML: a critical reading list
By Eirini Malliaraki, Feb 19 via tweet from @robmccargow
https://medium.com/@eirinimalliaraki/toward-ethical-transparent-and-fair-ai-ml-a-
critical-reading-list-d950e70a70ea

AppendixAdditional techniques and thoughts to continue
the conversation in your organization

Go beyond the
common,
simplistic
Example: Instead of a just tech
code of ethics…
consider a users’ bill of rights!

Example: Computer User’s Manifesto
● The user is always right. If there is a problem with
the use of the system, the system is the problem,
not the user.
● The user has the right to easily install software
and hardware systems.
● The user has the right to a system that performs
exactly as promised.
● The user has the right to easy-to-use instructions
for understanding and utilizing a system to
achieve desired goals.
● The user has the right to be in control of the
system and to be able to get the system to
respond to a request for attention.
● The user has the right to a system that provides
clear, understandable, and accurate information
regarding the task it is performing and the
progress toward completion.
● The user has the right to be clearly informed
about all system requirements for successfully
using software or hardware.
● The user has the right to know the limits of the
system’s capabilities.
● The user has the right to communicate with the
technology provider and receive a thoughtful and
helpful response when raising concerns.
● The user should be the master of software and
hardware technology, not vice-versa. Products
should be natural and intuitive to use.
By Dr. Claire-Marie Karat, as shared by Theo Mandel

Design principle User has a right …
Digestibility Not to work too hard to accomplish their goals
Digestibility To have their goals and their time respected
Clarity To understand exactly what they are getting
Clarity To be respected
Trust To be informed and in control
Familiarity To be oriented from the moment they start using a product
Familiarity To have their prior experience and knowledge respected
Delight To have an enjoyable experience
Delight To have an effortless (or appropriate effort) experience
Delight To have the UI disappear into the context of their work
Example: Design principles as user rights
Based on 5 simple UX principles from a blog post from InVision

Mindfulness is a journey,
not a destination

Discussions with the team
● Provide tools for mindfulness and self-awareness, build team awareness
● Acknowledge your own biases - be open to learn more
● Establish clear goals to understand biases
○ Why it matters to the team and to your work
● Establish communication ground rules
● Use ability to see biases of others with empathy and compassion
● Use these tools to approach the difficult conversations - but don’t shy away from them!

Integrating the tools for awareness
● Make the detection tools part of the design process – not once and done
● Define goals and checks around identified biases
○ Set objectives, check objectives
○ Key decision points, to avoid trouble
● Watch for emergent “excuses” for regressing to System 1 thinking

Using frameworks, guidelines, toolkits
● Cultural journey, not a formulaic process
● Tools can help with framing discussions initially, but not the endpoint
● Examples
○ Ethical OS
○ Big company’s ethical guidelines and programs
○ Google starting AI initiative and then ending it shortly after

Enabling communication
● Establish a culture of questioning
○ Listen for discord and regression
○ Bias buddies
○ Use technological aids where possible
● Communicate shared understanding continuously
● Be kind to each other

Organizational Questions
● Is there value in
○ defining a set of ethical guidance for employees?
○ providing guidance on techniques for identifying unintended negative consequences of work?
● Would different departments/products need different guidance?
● How might we
○ reflect our cultural norms in a code of ethics?
○ ensure we are helping people?
○ track progress?
● How can we describe the “lines” our work won’t cross?
● Would we include
○ mitigation strategies?
○ considerations of social consequences such as job displacement?
● Should we build on existing codes, guidance and best practices?

References

References, Page 1
AI Now Institute "After a Year of Tech Scandals, Our 10 Recommendations for AI. Let’s begin with better regulation,
protecting workers, and applying “truth in advertising” rules to AI" https://medium.com/@AINowInstitute/after-a-year-of-
tech-scandals-our-10-recommendations-for-ai-95b3b2c5e5
Angwin, Julia; Larson, Jeff; Mattu, Surya; and Kirchner, Lauren. "Machine Bias: There’s software used across the country
to predict future criminals. And it’s biased against blacks.“ ProPublica. 23 May 2016.
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Baer, Mark B. “The Connection Between Empathy Toward Others and Ethics,” Psychology Today.
https://www.psychologytoday.com/blog/empathy-and-relationships/201701/the-connection-between-empathy-toward-
others-and-ethics
Benson, Buster. “Cognitive bias cheat sheet.” 2016: https://betterhumans.coach.me/cognitive-bias-cheat-sheet-
55a472476b18
Beshears, J. Frederick, S. and Gino, F. “Test Yourself: Are You Being Tricked by Intuition?” Harvard Business Review.
2015: https://hbr.org/2015/04/test-yourself-are-you-being-tricked-by-intuition
Booch, Grady. “Ensure humans can unplug the machines.” TED@IBM. November 2016.
https://www.ted.com/talks/grady_booch_don_t_fear_superintelligence
Bowles, Cennydd. “Ethics in the AI Age,” presented at IxDA 2017. https://vimeo.com/album/3945251/video/209303271
Bowles, Cennydd. Future Ethics. 2018. NowNext Press.

References, Page 2
Bradberry, T. “13 Cognitive Biases That Really Screw Things Up For You.” 2018. https://www.linkedin.com/pulse/13-
cognitive-biases-really-screw-things-up-you-dr-travis-bradberry/
Campbell-Dollaghan, Kelsey. “The Year Dark Patterns Won.” Co.Design. 2016.
https://www.fastcodesign.com/3066586/the-year-dark-patterns-won
Christie, Gillian and Yach, Derek. ”Consider ethics when designing new technologies.” 2016.
https://techcrunch.com/2016/12/31/consider-ethics-when-designing-new-technologies/
"Data, Ethics, and AI: Practical activities for data scientists and other designers." IDEO Medium article.
https://medium.com/ideo-stories/data-ethics-and-ai-276723a1a2fc
Cutler, Adam; Pribić, Milena; and Humphrey, Lawrence. "Everyday Ethics for Artificial Intelligence: a Practical Guide for
Designers and Developers." IBM Medium Article. https://medium.com/design-ibm/everyday-ethics-for-artificial-
intelligence-75e173a9d8e8
Dastin, Jeffrey. "Amazon scraps secret AI recruiting tool that showed bias against women" Reuters. 9 October 2018.
https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-
showed-bias-against-women-idUSKCN1MK08G
Dempsey, Samantha and Taylor, Ciara. Designer’s Oath. http://designersoath.com/
DeNisco, Alison. “3 guiding principles for ethical AI, from IBM CEO Ginni Rometty.” Tech Republic. 17 January 2017.
http://www.techrepublic.com/article/3-guiding-principles-for-ethical-ai-from-ibm-ceo-ginni-rometty/

References, Page 3
Engber, Daniel. "Does the Trolley Problem Have a Problem? What if your answer to an absurd hypothetical question had
no bearing on how you behaved in real life?" Slate.com. 18 June 2018.
https://slate.com/technology/2018/06/psychologys-trolley-problem-might-have-a-problem.html
Foot, Philippa. Original paper introducing the Trolley Problem: "The Problem of Abortion and the Doctrine of the Double
Effect" Oxford Review, no. 5, 1967. http://www2.pitt.edu/~mthompso/readings/foot.pdf
Funchess, M. “Implicit Bias -- how it effects us and how we push through.” TEDxFlourCity. 2014.
https://www.youtube.com/watch?v=Fr8G7MtRNlk
Gray, D. Liminal Thinking: Create the change you want by changing the way you think. 2016. http://liminalthinking.com/
IDEO. "Data, Ethics, and AI: Practical activities for data scientists and other designers." IDEO Medium article.
https://medium.com/ideo-stories/data-ethics-and-ai-276723a1a2fc
Internet Encyclopedia of Philosophy, “Empathy and Sympathy in Ethics.” http://www.iep.utm.edu/emp-symp/
Hall, Erika. “The 9 Rules of Design Research.” 2018. https://medium.com/mule-design/the-9-rules-of-design-research-
1a273fdd1d3b
Halvorson, H.G. “How To Recognize (And Overcome) Your Unconscious Biases In Hiring.” Fast Company. 2015.
https://www.fastcompany.com/3043074/how-to-thwart-your-unconscious-biases-when-hiring-a-diverse-team
Judd, S. “Superfan.” 2018: http://www.sachajudd.com/superfan
Kahneman, Daniel. Thinking, Fast and Slow. 2013. Farrar, Straus and Giroux.

References, Page 4
Keereepart, T. “3 design principles to help us overcome everyday bias.” TEDxPortland. 2016.
https://www.youtube.com/watch?v=K6dstCUWsFY
Knight, Will. "Why does Beijing suddenly care about AI ethics?" MIT Technology Review.
https://www.technologyreview.com/s/613610/why-does-china-suddenly-care-about-ai-ethics-and-privacy/
Lawrence-Lightfoot, S. and Hoffmann Davis, J. The Art and Science of Portraiture. 1997.
http://www.saralawrencelightfoot.com/portraiture1.html
Madrigal, A.C. . “Disposable America.” The Atlantic. 2018.
https://www.theatlantic.com/technology/archive/2018/06/disposable-america/563204/
Maxwell, John C. Ethics 101: What Every Leader Needs to know. 2003. Center Street.
McIntosh, P. "White Privilege: Unpacking the Invisible Knapsack." Peace and Freedom Magazine. 1989.
https://nationalseedproject.org/white-privilege-unpacking-the-invisible-knapsack
Microsoft. “Keeping an Eye on AI with Dr. Kate Crawford.” Microsoft Research Podcast. 2018.
https://www.microsoft.com/en-us/research/blog/keeping-an-eye-on-ai-with-dr-kate-
crawford/?OCID=msr_podcast_kcrawford_tw
Pressner, K. ”Are you biased? I am.” TEDxBasel. 2016. https://www.youtube.com/watch?v=Bq_xYSOZrgU
Price, Rob. “Microsoft is deleting its AI chatbot's incredibly racist tweets” Business Insider UK. 24 March 2016.
http://www.businessinsider.com/microsoft-deletes-racist-genocidal-tweets-from-ai-chatbot-tay-2016-3

References, Page 5
Project Implicit. 2011: https://implicit.harvard.edu/implicit/index.jsp
Public Affairs. "Mortgage algorithms perpetuate racial bias in lending, study finds." UC Berkeley, Berkeley News. 13
November 2018. https://news.berkeley.edu/story_jump/mortgage-algorithms-perpetuate-racial-bias-in-lending-study-
finds/
Ratcliff, C. “12 ways to fight bias in your team” Synopsys of #uxchat on Twitter, hosted by K. Bachmann. 2018.
http://whatusersdo.com/blog/how-to-fight-bias-in-your-organisation-or-team/
Smith, S.E. “Why philosophy is so important in science education.” Quartz. 2017: https://qz.com/1132948/why-
philosophy-is-so-important-in-science-education/
Soll, J.B., Milkman, K.L. and Payne, J.W. “Outsmart your own biases.” Harvard Business Review. 2015.
https://hbr.org/2015/05/outsmart-your-own-biases
#uxchat on Twitter “Embedding Ethics into Design,” hosted by What Users Do. 2018.
http://whatusersdo.com/blog/embedding-ethics-design-uxchat-roundup/
Velasquez, Manuel; Andre, Claire; Shanks, S.J. Thomas; and Meyer, Michael J. "What is Ethics?" Markkula Center for
Applied Ethics. https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/what-is-ethics/
Vincent, James. "Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech. Nearly three years
after the company was called out, it hasn’t gone beyond a quick workaround." The Verge. 12 January 2018.
https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai

References, Page 6
Wachter-Boettcher, Sara and Meyer, Eric. Design for Real Life. 2016. https://abookapart.com/products/design-for-real-life
Watcher-Boettcher, Sara. Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech. 2017.
http://www.sarawb.com/technically-wrong/
@sara_ann_marie. “the line from "look at our cool new facial-recognition app" to "wow this thing only works for white
people" is a fucking millimeter long and clearly marked. why can't a single tech co seem to be able to see it?” 2018.
https://twitter.com/sara_ann_marie/status/953783812315602944
Wendt, Thomas. "Empathy as Faux Ethics.” 2017. https://www.epicpeople.org/empathy-faux-ethics/

Designing more ethical and unbiased experiences

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Similar to Designing more ethical and unbiased experiences

Similar to Designing more ethical and unbiased experiences (20)

More from Karen Bachmann

More from Karen Bachmann (10)

Recently uploaded

Recently uploaded (19)

Designing more ethical and unbiased experiences