From recruitment to car driving, we all know success and failures examples with AI. Let's focus on failing cases due to your own societal biases and how to taggle these new challenges
7. 7
Job posts gathered into a single position: prone to many biases. Data biases and Model biases
What do you think of combining : architect + dev+ testing + maintenance ?
The nature of AI into the work market
Data Scientist
Data Analyst
Data Engineer
Machine Learning Scientist
Machine Learning Engineer
Statistician
Find, clean and organize data for companies
Transform and manipulate large datasets to suit the analysis of companies
Perform batch processing or real time processing on gathered/stored data
Research new data approaches and algorithms
Create data funnels and deliver software solutions
Interpret, analyze and report statistical information
8. 8
● Bias (in statistics) : “the difference between the expectation of the sample estimator and the true population value, which
reduces the representativeness of the estimator by systematically distorting it”
● “The big takeaway is that we don’t know what we don’t know,” (Alice Popejoy)
What do you mean about biases ?
11. 11
● MIT study decision-making of self-driving car on
killing scenario
● Scenario with
○ high/low level of education
○ young/old
○ male/female
○ pets
○ traffic signal respect
http://moralmachine.mit.edu/
The Moral Machine
12. 12
● Results “rank” you on
○ number of death
○ law respect
○ gender,age,health, ...
● Need ethical choices at the government,
insurance ? manufacturer ? passengers ?
The Moral Machine
13. 13
● 3 different clusters
● Cultural differences
○ USA/Europe kill the oldest vs Japon saves the oldest
○ Colombia saves high educated people vs Finland doesn’t matter
○ South america/France saves women
The Moral Machine experiment, Edmond Awad et al. (2018), Nature
The Moral Machine
14. 14
Ethics in clinical investigations
● Technical committee to review scientific foundations and safety (in France, ANSM)
● Ethics committee on animal AND humans (in France, CPP)
○ Composed by medical professional and citizen
○ Review application form on
■ benefit/risk
■ information quality
■ resources to conduct the study
■ patient recruitment process
■ Patient consent modality
○ Follow the experiment process
15. 15
Applying ethics committee in AI at Google
● Advanced Technology External Advisory Council (ATEAC)
○ Ensure the white paper principles for AI at Google
■ Be socially beneficial
■ Avoid creating or reinforcing unfair bias
■ Be built and tested for safety
■ Be accountable to people
■ Incorporate privacy design principles
■ Uphold high standards of scientific excellence
■ Be made available for uses that accord with these principles
● Dissolved 1 week after creation
○ Ethics of some attendees were discussed …
○ Questioning themselves of the needs to represent every part of the society
17. 17
● In the 90s, “I was breaking all of [my classmates’] facial-recognition software
because apparently all the pictures they were taking were of people with
significantly less melanin than I have” (Charles Isbell)
● In 2015, the “gorilla mistake” in Google Photos
● 25 years, not the same learning model at all, but the same root cause
Face recognition from the 90s to nowadays
18. 18
● In 2014, Amazon develops an IA to “find” key success factor and hire these people
● Train on their 10 last years hired people
● 89% of the engineering workforce is male
● At the end, you have sexist recruitment AI !
● Representative data inside the company by not inside the society …
AI recruitment by Amazon
19. 19
Biases in the Features
Outcome outputFeatures
Model
Diploma Gender Hobbies
Nationality University ...
HIRED ?
Model
Outcome output
Features
● Lack of an appropriate set of features
● Lack of an appropriate dataset
● Imbalanced dataset or bias in the output
● Unawareness: remove sensitive features from your
data
20. 20
Tutorial: your first bias detector
Survival of passengers on the Titanic
● Decision Tree
● Leaves = class labels
● Nodes = splitting/conjunctions of features
● Important nodes = less deep, lot of observations
● Many Framework coexist:
FairML, “What If”, IBM Bias Assessment
● ‘No universal solution’: combine them
Summarize: Your chance of survival were
good iff:
- you were a woman
- you were a young boy with few siblings
21. 21
● Bad advice pointed out by IBM in internal in 2017
○ ex : Suggest a cancer patient with severe bleeding be given a drug that could cause the bleeding to worsen
● Started off by using real patient data
● Fed it with hypothetical data
● “Synthetic cases allow you to treat and train Watson on a variety of patient variables and conditions that might not be
present in random patient samples, but are important to treatment recommendations” (Edward Barbani)
● Pointed out the difficulties to collect representative data in medical
Unsafe medical recommendations by IBM Watson
22. 22
Data Augmentation
● Historically for images: rotation, flipping, adding noise…
● Object detection models : performance loss on corrupted images
● CNNs generalize poorly to novel distortion types, despite being trained on a variety of other distortions
23. 23
Generating new data with GANS
source: Medium, “Generative Adversarial with Maths” by Madhu
Sanjeevi
Discriminator
Generator
Generator
24. 24
Understanding biases and Generalization in Deep
Generative Models
Can we learn the distribution of data with GANS ?
What are the inductive biases of Deep Generative models ?
● Unbiased and consistent density estimation impossible
● Inductive biases
● Similar cognitive bias as humans: numerosity
● Weber’s law: relative change (ratio)
True data Generated data
28. 28
Group Fairness
● Demographic Parity
○ ex : gender independent
● Equal odds
○ ex : take into account the reality
statistics in one side
● Equal opportunity
○ ex : take into account the reality
statistics in both side
33. 33
Is it working ?
Examples of Explanations are usually cherry-picked !
34. 34
Individual Fairness for Explainability
The fact that 2 resulting saliency maps are different is fundamentally due to the network itself being fragile to
such perturbations
37. 37
● Experienced lane recognition algorithm
● Doesn’t expect that 3 stickers break the robustness
● Hacking the AI with thinks that doesn’t matter humans
Tesla autonomous driving hacked with stickers
39. 39
Hacking tutorial: design your own adversarial example
CAT DOG CAT DOG
if B is predicted as
a cat, retry to cross
the decision
boundary with B !
Linear Model NON Linear Model
A
A
B
B
40. 40
Hacking Tutorial: I don’t have access to the model
This is not a
car !
This is not a
car !
41. 41
Defending against adversarial attacks ?!
SUCCESS
FAILURES
2013
Discovery of
adversarial examples
2015
Fast Gradient
Sign
2015
DeepFool
2016
Carlini Wagner
2019
Unforeseen
Attack
2013
Adversarial Training
BRUTE FORCE: training with
adv
2015
Defensive Distillation
output probabilities rather than hard
decisions
2016
JPEG compression
against adv
42. 42
Why is it hard to defend against adversarial examples?
● Adaptative: block one type of attacks but leaves vulnerability open to an attacker who knows the defense being used
● Hard to defend because hard to understand the theory ?
43. 43
Adversarial Examples are not Bugs, they are
features
BIRD
DOG
CAT
“Bird”
“Dog”
“Cat”
create the label-target adversarial dataset
DOG
CAT
BIRD
“Dog”
“Cat”
“Bird”
46. 46
What about in the future ?
● Art project exposed racial bias in the biggest image dataset
ImageNet in Sept. 2019
● ImageNet will remove 600,000 images
● Number of publications per year with the name Imagenet in
the abstract…