Etsy is an online bazaar for handmade goods, and one for the key aspects for the site itself is the Etsy communities and sub-communities that have and continue to form and grow. The Etsy development community is the group of smart engineers that grow and maintain the site that helps to facilitate these amazing Etsy communities. While these engineers are very smart, you should recall the old adage, "A stupid person is someone who never learns from his mistakes, a smart person is someone who learns from his mistakes, and a wise person is someone who learns from other mistakes." At Etsy we embrace learning from mistakes and sharing with others so that they may learn from these mistakes as well because we also know that no one is perfect. Here, I will share several of the bumps along the way that took us from being small to large and still growing with more community and emphasis on culture, all while maintaing that small startup feeling.
9. Developers - Write Code
DBAs - Write SQL, Stored Procedures
Ops - Deploy Code and Maintain Production
10. “Any organization that designs a system
(defined broadly) will produce a design whose
structure is a copy of the organization’s
communication structure.”
— Melvin Conway, 1968
11. Developers - Write Code
DBAs - Write SQL, Stored Procedures
Ops - Deploy Code and Maintain Production
12. Sprouter
aka Stored Procedure Router
Per Web Server on Port 8010
Maps name/arguments to Postgres stored procedure
Cache
Supports Sharding (Theoretically)
Goal — Scale the Web Site
13. Forcibly Centralizes Database Access
Hides Data Store Implementation
“Clever” Automatic Caching
Prevents Developers from Writing SQL
15. Life and Death of Sprouter
Fall ’07 First Discussed
Spring ’08 Alpha Version Released
Fall ’08 Released in Production
Spring ’09 Sprouter Deprecated
Spring ’11 Sprouter is Dead!
21. How Do We Organize?
Ops — Five People
Search — Five People
The Rest??
25. Re p o e
r a te u c e n
S e p a o l r, L
Ja v a, S
epo epo
rm R a te R p l o y
P l at f o MP S e p a r t De
LA
Di f fe re n
epo
Re p o Re p o ate R oDB
e p a r o ng
r a te s s S e p a r a te S
S e p a rdp re u by l a, M
Wo R Sc a
26. What? Why?
Too Big to be One Team
SVN Repository was Too Big to be Just One
Solution: Split the Workforce, Split the Repository
28. SVN to Git in 1,000 Easy
Git Fans using git-svn
Git is Better Equipped to Handle the Mammoth Repository
Merge the Repositories Together
Devise a Workflow, Document It, Teach It
Do the Switch!
30. Divide and Merge
Break-up By Customer or Necessary Technological Difference
Create Constantly Changing Sub-Teams within Divisions
Make it Easy to Move Between Focus Areas
31. or k , e tc.
t ac k Ex pe rt ise
Jav a
S
a re , Ne t w
H a rd w
ome r
C us t ome r C us t
e the
e the rs a r
rs a r B u ye
Se l le
C us t ome r
u i re d Si lo the
y Re q g is
Le gall ngi n eerin
E
34. Continuous... Unceremoniously... Initiated By...
Push Changes to
Deployment Anyone
Production
Apply Quality
Integration Anyone
Process
Release New
Delivery Product
Features
35. Continuous... Unceremoniously... Initiated By...
Push Changes to
Deployment Anyone
Production
Apply Quality
Integration Anyone
Process
o w? Release New ???
H Delivery Product
Features
38. Possible Takeaways
Look Back on History to Avoid Repeating the Bad Parts
Cultivating Culture Requires a Constant Concerted Effort
Don’t Optimize Too Early
Keep It Simple
Handmade, Vintage, Supplies\nHow Founded...\n15 Million Members\n>800,000 shops\n$525.6 Million in sales in 2011\n
My First Etsy 2006\nMy Wedding Dress 2010\nMy First Halloween at Etsy\n
Community at Large\nTop Tips for Etsy Sellers include:\nTeams!\nHome for the Holidays\n
\n
\n
This is taking off\nEngineers excited to engineer\n
2005 only 3 Guys\nBoing, Boing August 4 Guys\n2006 and 2007... something like\n
Conway’s Law\n
\n
Promises\nAbstraction can buy you flexibility, maybe this did\n
The Good\n
The Bad\n
Took a Year to implement\nTwo Year’s to take down\n
What Changed?\nCulture Shift\nChad Dickerson\nInvolved several engineers leaving\n
Next battle\n
Before alluded to Four Guys\n
Partitioned roles that led to Sprouter\n
15 to 30 to 60\n5 people in the 3 partions\nbut 20?\n
Ops - Hardware\nSearch - Acquired Adtuitive, Java stack\nThe other 50?? and growing??\nHow to organize is a question that plagues every company at some point\n
Can’t Just have a cloud\n
First crack\n20 per team also hard\n5 to 7 is an ideal team size\n
Something like this while we were at 40-50\n
But we went further\n
\n
All depended on the core libraries anyways\nWrapper tools\n“Smelled of unicorn”\n
Blog post on codeascraft.com\n
Current attempt looks something like this\n-next-\n
\n
\n
2010 DevOps and Actual Continuous Deployment\nWhat we normally talk about...\n
Our commandments\n
What I usually explain\n
Done with first Two\nHow to get the third\n
I showed this earlier\nProduct is the third piece of the puzzle\nIncludes product management, design, quality\n
Growing a community with the product\nGrowing a community structure similarly inside to maintain and create more\nThis was about learning lessons\n
Possible Takeaways\n
Visit Etsy, checkout the 60 pages worth of local artisans\nSee how code is our craft on the product engineering side\nView our previous slide decks for more information on these topics\n