Video and slides synchronized, mp3 and slide download available at http://bit.ly/12Xi2lC. Wil Stuckey explains how Etsy manages to deploy nearly ~10,000 changes in one year, and how they run A/B experiments in the midst of continual code change.Filmed at qconnewyork.com. Wil Stuckey is Software Engineer at Etsy.

1. Wil Stuckey
@quiiver

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/etsy-deploy

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

7. 25 Million Members

8. 25 Million Members
900,000 Shops
`

9. 25 Million Members
900,000 Shops
1.4B Page Views / Month

10. Managing
Experimentation

11. Managing
Experimentation

12. Managing
Launches

13. 1. Experiments
2. Rampups
3. Communications

14. Life Before Continuous
Deployments

15. Initial Idea &
Planning

17. Geocities-ize

18. Initial Idea &
Planning
Design &
Development

20. Initial Idea &
Planning
Deploy &
Launch!
Design &
Development

21. Pain?

22. Testing?

23. Communication?

24. How can we do better?

25. Living in a
Continuous
Deployment World

26. Total Deploys: 4,040
Total Commits: 45,447
Total Unique Authors: 220
2013 ... So far

27. Total Deploys: 4,040
Total Commits: 45,447
Total Unique Authors: 220
Average Deploys per day: 35.75
Average Authors per deploy: 7.01
Average Commits per deploy: 11.25
2013 ... So far

28. That’s a lot of people

29. We are only halfway
through the year!

30. CHAOS

31. Config System
$server_config['my_awesome_feature'] = [
'enabled' => 10,
'admin' => 'on',
'groups' => [1234],
'users' => ['wstuckey', 'dmckinley'],
];

32. Branching in Code
if (Feature::isEnabled('my_awesome_feature')) {
// display my_awesome_feature...
} else {
// continue on
}

33. Typical Launch
Cycle

34. Initial Idea &
Planning

35. Initial Idea &
Planning
Design &
Development
Launch!

36. Initial Idea &
Planning
Design &
Development
Launch!
Internal Admin Launch

37. Initial Idea &
Planning
Design &
Development
Launch!
Internal Admin Launch
Public Prototype

38. Initial Idea &
Planning
Design &
Development
Launch!
Internal Admin Launch
Public Prototype
50% Experiment

39. • Small iterative changes

40. • Small iterative changes
• Product exists in prod as soon as possible

41. • Small iterative changes
• Product exists in prod as soon as possible
• Can be available for small groups for
testing and feedback.

42. • Small iterative changes
• Product exists in prod as soon as possible
• Can be available for small groups for
testing and feedback.
• Easy to communicate milestones because
they happen as part of the development
process.

43. Because of the
constant change you
have to constantly
communicate.

44. Start Simple

45. Wiki + Email

46. • Works great in the
beginning
Wiki

47. • Works great in the
beginning
• Quickly becomes
overwhelming as you
increase your
concurrent launches.
Wiki

48. • Works great in the
beginning
• Quickly becomes
overwhelming as you
increase your
concurrent launches.
• Requires manual labor
to keep stakeholders
informed
Wiki

49. FRUSTRATION

50. We can do better!

51. Frank Harris
@hirefrank

52. Launch Calendar

53. Launch Calendar
• Simple web app
• Collect structured meta-data about the
launch
• Central location for upcoming, current
and past launch information

54. Launch Calendar
Collect launch specific meta-data

55. Launch Calendar
• Owner
• Launch Date
• Internal Launch Date
• Team
• Expected Reaction
Collect launch specific meta-data

56. Launch Calendar
✉
Make your email work for you

57. ✉
Launch Calendar
• Automated emails to product owners
around deadlines
• Daily emails to product groups containing
upcoming and recently launched
launches
Make your email work for you

58. Launch Calendar
Make your email work for you
✉

59. Launch Calendar
• Highly utilized across the organization
• During the year+ that it was running we
recorded and tracked over 275 launches
How did it go?

60. “This beats the crap out
of the wiki page in every
conceivable way.” - real
feedback

61. What could we do better?

62. What would a more
integrated product look
like?

63. Can we automate more?

64. We can do better!

66. Catapult
A unified launch management tool.

68. “I believe that by changing
the ‘add to cart’ button to
blue we will increase
conversion rate.”

69. Catapult

72. $server_config['my_awesome_button'] = [
'enabled' => 50,
'admin' => 'on',
];
Remember this?

76. Hack, Deploy, Hack, Hack
Deploy!

83. Integration is
awesome.

84. Communication is
awesome.

85. Start Simple.

86. Build process that
enable you to Ship.

87. Never stop
deploying!

88. Thank You!

89. Wil Stuckey
@quiiver