Consensual Software: Prioritizing Trust & Safety

@tsunamino
!
"
Consensual Software:
Prioritizing Trust & Safety

InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations
/github-consensual-software
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

Consensual Software: PrioritizingTrust & Safety! @tsunamino 2
@tsunamino
http://www.wocintechchat.com/blog/wocintechphotos

@tsunamino

😬

@tsunamino

😬

Danielle Leong
Engineer on GitHub’s Community & Safety team @tsunamino

GitHub is a web-based version-control
and collaboration platform for
software developers.
@tsunamino

Collaboration in text form is hard.

Build systems that empower inclusive
and healthy communities and
encourage good online citizenship.
Discourage behavior that is destructive
or threatens personal safety.

Some things we’ve built
• Repository Invitations
• User blocking API
• Organizations have the ability to block users
• Contributor badges
• Community Health management tools
• Transparency around comment edit history

On any platform, any time you have
user-to-user interactions you have
the possibility of harassment.

Types of online harassment
• Posting personal info (doxing)
• Hate speech
• Sexual or pornographic content
• Impersonation
• SWATing
• Spam
• Identity theft
• Exclusionary behavior
• Death and rape threats
• Dog piling
• Personal attacks
• Threatening and inappropriate images
• So many more

Racist repository names led to
repository invitations

Women, women of color, and trans
women experience harassment daily.
We are experts in online safety.
Not to say that men don’t experience harassment, but women tend to get the worst of it

70% women

40% women of color

30% trans women

30% white men

4 engineers

1 designer

1 product manager

1 qa

1 year of work to address 7 years of backlog

Some tactics women use to avoid
online harassment
• Not using full names on social media sites
• Not using real photos for avatars
• Turning oﬀ location tracking data
• Regularly Googling name, address, & phone numbers
• Using a burner phone number service & PO boxes
• Turning oﬀ direct messages
• Not live posting on social media during vacation

@tsunamino

It’s our job to write software that
protects vulnerable people.

http://www.huﬃngtonpost.com/aran-khanna/stalking-your-friends-on-facebook-messenger-_b_7452912.html

http://www.huﬃngtonpost.com/aran-khanna/venmo-money_b_8418130.html

""
" "💥
💥
BUT FREE SPEECH
BUT FREE SPEECH

https://www.ﬂickr.com/photos/epw/348448681/in/photolist-wMTDT-eMnoZK-ahqLSQ-dWv9vQ-dK4Hnd-eLPF8d-DKNYyX-jnCGNf-eNgckU-388Mus-eKGUs3-
eR26Ui-58nGwR-idg5CJ-9Jwyxb-639vTn-FmvThd-4e5Fﬀ-ah1Dse-eiTkAT-9Jwyzh-pvMWjr-ovCHpM-dWtVVp-i2aAxc-5mDSDM-DrXkrE-eciQvg-c9811Y-4t5Tcr-dbzN7K-
r1WnoK-qKrNny-qKrueE-dWtJ4T-hYCB6D-hDr4y9-bq3RAJ-92ee1-dTuCwi-Ji54Ed-eMa6Du-9yqLWZ-7PoVXq-b2Ev8R-aho1T2-9TR1s8-9E6usg-nzmYLL-dNEzjC

“Free speech” does not mean you get
to say whatever you want without
consequences

I mean, you can, but it’ll cost you a bunch of money

http://www.businessinsider.com/twitter-trolls-caused-salesforce-to-walk-away-from-deal-2016-10

http://sﬁst.com/2016/10/18/disney_dropped_twitter_bid_because.php

https://www.bloomberg.com/news/articles/2016-10-17/disney-said-to-have-dropped-twitter-pursuit-partly-over-image

http://money.cnn.com/2016/10/14/technology/salesforce-not-buying-twitter/

https://www.theguardian.com/commentisfree/2017/jan/03/ive-left-twitter-unusable-anyone-but-trolls-robots-dictators-lindy-west

More than half of app users have
uninstalled or decided to not install an
app due to concerns about personal
information
Pew Research Center
http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/

Safety features shouldn’t be an
afterthought. This leads to years of
technical debt.

70% women

40% women of color

30% trans women

30% white men

4 engineers

1 designer

1 product manager

1 qa

1 year of work to address 7 years of backlog

Design software with trust & safety in
mind before it becomes a problem.

Have a deﬁnition of harassment.

GitHub’s deﬁnition of harassment is
destructive behavior that undermines
productivity.

While it is not forbidden to broach topics such
as age, body size, disability, ethnicity, gender
identity and expression, level of experience,
nationality, personal appearance, race, religion,
or sexual identity and orientation, we do not
tolerate speech that attacks a person or group of
people on the basis of who they are.

https://mobile.twitter.com/TweetAnnaMarie/status/789957313649967104

Build Trust & Safety into new features
• Ensure explicit consent
• Keep an audit log trail
• Strive to minimize abuse
https://githubengineering.com/community-and-safety-feature-reviews/

Build consensual software.

🤔

Consent means all parties are aware and
able to agree to an interaction.

https://www.youtube.com/watch?v=oQbei5JGiT8

Consensual software means all users
are aware and explicitly consenting to
every interaction and disclosure of
their personal data on your site or
service.

More than half of app users have
uninstalled or decided to not install an
app due to concerns about personal
information
Pew Research Center
http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/

%&
💬

Ensure explicit consent

%
• Does Alice want to be in
this interaction?
• Is it safe to block Bob?
• How much of Alice’s
information is revealed
if she reports Bob?

🍆

https://www.vg247.com/2015/06/03/the-sad-saga-of-lego-universes-dong-detection/

https://techcrunch.com/2016/03/24/microsoft-silences-its-new-a-i-bot-tay-after-twitter-users-teach-it-racism/

https://qz.com/818151/virtual-assistant-bots-like-siri-alexa-and-cortana-spend-much-of-their-time-fending-oﬀ-sexual-harassment/

• Is it easy to create sock
puppet accounts?
• Can you introduce
friction to discourage
destructive behavior?
• Is it easy to learn from
mistakes?
&

Keep an audit log trail

)
• Is there an audit log for
your support folks?
• If you can edit content,
do you know who did
what change?
• How many people have
reported Bob?

Strive to minimize abuse

*
• Are you limiting 0-day
account activity?
• Are you rate limiting
potentially abusive
accounts?
• If you have a reputation
score, how is it
calculated?

https://githubengineering.com/community-and-safety-
feature-reviews/

Smart Girl’s Guide to Privacy
by Violet Blue
HATERS: Harassment, Abuse, and Violence
Online
by Bailey Poland
The Internet of Garbage
by Sarah Jeong

Have a deﬁnition of harassment.

Trust & Safety guidelines for new features
• Ensure explicit consent
• Keep an audit log trail
• Strive to minimize abuse
https://githubengineering.com/community-and-safety-feature-reviews/

Make sure your users are aware and
explicitly consenting to every
interaction and disclosure of their
personal data on your site or service.

Hire more women of color and trans
women. We are experts in avoiding
online harassment.

Questions?

How people build software
!
"
Thanks!

Watch the video with slide synchronization on
InfoQ.com!
https://www.infoq.com/presentations/github-
consensual-software

Consensual Software: Prioritizing Trust & Safety

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de C4Media

Mais de C4Media (20)

Último

Último (20)

Consensual Software: Prioritizing Trust & Safety