SlideShare uma empresa Scribd logo
1 de 32
Baixar para ler offline
Engineering Change:
   Enable rapid experimentation



                       Kellan Elliott-McCrea
                       @kellan
                       CTO, Etsy



Thursday, May 17, 12
* A marketplace for handmade and vintage goods.

                  In April 2012

                  * 875,000 artisans and creative independents
                  * $63.6 million USD in sales
                  * 1.32 billion page views




Thursday, May 17, 12
Engineering Change:

       * Continuous Deployment
       * Metrics driven development




Thursday, May 17, 12
Engineering Change:

       * Make mistakes quickly
       * Learn from mistakes




Thursday, May 17, 12
Engineering Change:

       * Continuous Deployment
       * Metrics driven development




Thursday, May 17, 12
Continuous Deployment:


                       a technique for
                       rapid iteration


Thursday, May 17, 12
Continuous Deployment:


                   make failure cheap



Thursday, May 17, 12
Continuous Deployment:


                        small changes,
                       pushed frequently


Thursday, May 17, 12
Continuous Deployment:

                         small changes,
                       pushed frequently.
                         To production.

Thursday, May 17, 12
Continuous Deployment:

                       10,068 releases to
                       production in 2011
                            @ Etsy

Thursday, May 17, 12
Continuous Deployment:


                       How?



Thursday, May 17, 12
Continuous Deployment:
                                  No branching.

       “All existing revision control systems were
       built by people who build installed
       software”
       - Paul Hammond,
       Always Ship Trunk, Velocity 2010
       Thursday, March 17, 2011




Thursday, May 17, 12
Continuous Deployment:

                        feature flags
                if ($cfg[‘awesome_new_search’]) {
                    # new hotness
                    $rsp = do_solr();
                } else {
                    # boring old stuff
                    $rsp = do_grep();
                }



Thursday, May 17, 12
Continuous Deployment:
                       Ramp - ups
                       (on top of feature flags)


         1. Launch to staff only
         2. Launch to 1% of all users
         3. Launch to members of a beta group




Thursday, May 17, 12
Continuous Deployment:


                       any engineer can launch a feature to

                       1% of users


Thursday, May 17, 12
Continuous Deployment:


           ~200 experiments
           live right now


Thursday, May 17, 12
Metrics driven development:

       optimize for learning




Thursday, May 17, 12
Metrics driven development:

       measure everything




Thursday, May 17, 12
Metrics driven development:

       getting started:
       make it easy



Thursday, May 17, 12
Metrics driven development:

       measure everything:
       logs, monitoring, A/B, graphs,
       StatsD




Thursday, May 17, 12
Metrics driven development:

       StatsD @
       Instagram, Pinterest, Github,
       Mozilla, LAN.com, Zynga,
       Kickstarter, LivingSocial and 70+
       other companies


Thursday, May 17, 12
Metrics driven development:
       StatsD
       “We love statsd at Instagram. Written by Etsy, it’s a
       network daemon that aggregates and rolls-up data into
       Graphite. At its core, it has two types of statistics: counter
       and timers. We use the counters to track everything from
       number of signups per second to number of likes, and we
       use timers to time generation of feeds, how long it takes to
       follow users, and any other major action.”
       - Mike Krieger, Founder and CTO, Instagram.




Thursday, May 17, 12
Metrics driven development:

       StatsD
       Open source,
       a daemon for timing and counting things,
       graphs rates, averages and 95th percentile,
       20-30 minute install,
       easy to use



Thursday, May 17, 12
Metrics driven development


    StatsD::timing("page.render", $msec);




Thursday, May 17, 12
Metrics driven development




Thursday, May 17, 12
Metrics driven development

       getting started:
       your 5 core metrics



Thursday, May 17, 12
Metrics driven development

       5 core metrics @ Etsy:
       sign ups, logins, checkout, new
       listings, posts in the bugs forums




Thursday, May 17, 12
Metrics driven development

       378,000 metrics,
       collected roughly once a
       second



Thursday, May 17, 12
Metrics driven development

       the confidence to
       change



Thursday, May 17, 12
Engineering change:
   Rapid experimentation
                Adapt, find market fit.




Thursday, May 17, 12
Engineering change:
   Rapid experimentation
                Adapt, find market fit,
                addictive to engineers.




Thursday, May 17, 12
Thank you!



                       Lean more: http://codeascraft.etsy.com



Thursday, May 17, 12

Mais conteúdo relacionado

Destaque

Engineering Revisions and Engineering Change Orders (ECO)
Engineering Revisions and Engineering Change Orders (ECO)Engineering Revisions and Engineering Change Orders (ECO)
Engineering Revisions and Engineering Change Orders (ECO)Rootstock Software
 
Boeing Insitu's Enterprise PLM Journey with Aras
Boeing Insitu's Enterprise PLM Journey with ArasBoeing Insitu's Enterprise PLM Journey with Aras
Boeing Insitu's Enterprise PLM Journey with ArasAras
 
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...Rootstock Software
 
Engineering change management webinar april 2013
Engineering change management webinar april 2013Engineering change management webinar april 2013
Engineering change management webinar april 2013John Cachat
 
Change Management
Change ManagementChange Management
Change ManagementEstragon
 
Engineering Change Management - Overview and Best Practices
Engineering Change Management - Overview and Best PracticesEngineering Change Management - Overview and Best Practices
Engineering Change Management - Overview and Best PracticesShobhit Singhal
 
Change management strategy ppt
Change management strategy pptChange management strategy ppt
Change management strategy pptsonips
 
Change Management PPT Slides
Change Management PPT SlidesChange Management PPT Slides
Change Management PPT SlidesYodhia Antariksa
 

Destaque (9)

Engineering Revisions and Engineering Change Orders (ECO)
Engineering Revisions and Engineering Change Orders (ECO)Engineering Revisions and Engineering Change Orders (ECO)
Engineering Revisions and Engineering Change Orders (ECO)
 
Boeing Insitu's Enterprise PLM Journey with Aras
Boeing Insitu's Enterprise PLM Journey with ArasBoeing Insitu's Enterprise PLM Journey with Aras
Boeing Insitu's Enterprise PLM Journey with Aras
 
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...
ERP 101 Series: Engineering Basics - The Importance of Part Master Records an...
 
Engineering change management webinar april 2013
Engineering change management webinar april 2013Engineering change management webinar april 2013
Engineering change management webinar april 2013
 
SAP Engineering Change Management(ECM)
SAP Engineering Change Management(ECM) SAP Engineering Change Management(ECM)
SAP Engineering Change Management(ECM)
 
Change Management
Change ManagementChange Management
Change Management
 
Engineering Change Management - Overview and Best Practices
Engineering Change Management - Overview and Best PracticesEngineering Change Management - Overview and Best Practices
Engineering Change Management - Overview and Best Practices
 
Change management strategy ppt
Change management strategy pptChange management strategy ppt
Change management strategy ppt
 
Change Management PPT Slides
Change Management PPT SlidesChange Management PPT Slides
Change Management PPT Slides
 

Semelhante a Engineering Change

How we killed our process, technology stack, and assumptions – and survived
How we killed our process, technology stack, and assumptions – and survivedHow we killed our process, technology stack, and assumptions – and survived
How we killed our process, technology stack, and assumptions – and survivedJason Lotito
 
Smoothing the Continuous Delivery Path - A Tale of Two Teams
Smoothing the Continuous Delivery Path - A Tale of Two TeamsSmoothing the Continuous Delivery Path - A Tale of Two Teams
Smoothing the Continuous Delivery Path - A Tale of Two TeamsEqual Experts
 
Optimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceOptimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceKellan
 
Building Rich User Experiences w/o JavaScript Spaghetti
Building Rich User Experiences w/o JavaScript SpaghettiBuilding Rich User Experiences w/o JavaScript Spaghetti
Building Rich User Experiences w/o JavaScript SpaghettiJared Faris
 
How We Killed Our Process, Technology Stack, and Assumptions — and Survived
How We Killed Our Process, Technology Stack, and Assumptions — and SurvivedHow We Killed Our Process, Technology Stack, and Assumptions — and Survived
How We Killed Our Process, Technology Stack, and Assumptions — and Survivedcolleenfry
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous DeploymentBrian Henerey
 
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...DataKitchen
 
Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013500 Startups
 
Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps Gene Kim
 
Make it Real: Designing with Data
Make it Real: Designing with DataMake it Real: Designing with Data
Make it Real: Designing with DataUXPA Boston
 
Tech fuse11 toolingtestingci-vs2010teamcity
Tech fuse11 toolingtestingci-vs2010teamcityTech fuse11 toolingtestingci-vs2010teamcity
Tech fuse11 toolingtestingci-vs2010teamcityBaskin Tapkan
 
Data Science: Good, Bad and Ugly by Irina Kukuyeva
Data Science: Good, Bad and Ugly by Irina KukuyevaData Science: Good, Bad and Ugly by Irina Kukuyeva
Data Science: Good, Bad and Ugly by Irina KukuyevaData Con LA
 
How I ended up touching Magento core
How I ended up touching Magento coreHow I ended up touching Magento core
How I ended up touching Magento coreAlessandro Ronchi
 
Fri benghiat gil-odsc-data-kitchen-data science to dataops
Fri benghiat gil-odsc-data-kitchen-data science to dataopsFri benghiat gil-odsc-data-kitchen-data science to dataops
Fri benghiat gil-odsc-data-kitchen-data science to dataopsDataKitchen
 
ODSC data science to DataOps
ODSC data science to DataOpsODSC data science to DataOps
ODSC data science to DataOpsChristopher Bergh
 
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"PROIDEA
 
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology OrgsWhy Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology OrgsGene Kim
 
Continuous delivery agile_2012
Continuous delivery agile_2012Continuous delivery agile_2012
Continuous delivery agile_2012drewz lin
 

Semelhante a Engineering Change (20)

How we killed our process, technology stack, and assumptions – and survived
How we killed our process, technology stack, and assumptions – and survivedHow we killed our process, technology stack, and assumptions – and survived
How we killed our process, technology stack, and assumptions – and survived
 
Smoothing the Continuous Delivery Path - A Tale of Two Teams
Smoothing the Continuous Delivery Path - A Tale of Two TeamsSmoothing the Continuous Delivery Path - A Tale of Two Teams
Smoothing the Continuous Delivery Path - A Tale of Two Teams
 
Optimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceOptimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerce
 
Building Rich User Experiences w/o JavaScript Spaghetti
Building Rich User Experiences w/o JavaScript SpaghettiBuilding Rich User Experiences w/o JavaScript Spaghetti
Building Rich User Experiences w/o JavaScript Spaghetti
 
How We Killed Our Process, Technology Stack, and Assumptions — and Survived
How We Killed Our Process, Technology Stack, and Assumptions — and SurvivedHow We Killed Our Process, Technology Stack, and Assumptions — and Survived
How We Killed Our Process, Technology Stack, and Assumptions — and Survived
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous Deployment
 
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
Strata+hadoop data kitchen-seven-steps-to-high-velocity-data-analytics-with d...
 
April JavaScript Tools
April JavaScript ToolsApril JavaScript Tools
April JavaScript Tools
 
Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013Ross Snyder, Etsy, SXSW Lean Startup 2013
Ross Snyder, Etsy, SXSW Lean Startup 2013
 
Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps
 
Make it Real: Designing with Data
Make it Real: Designing with DataMake it Real: Designing with Data
Make it Real: Designing with Data
 
Tech fuse11 toolingtestingci-vs2010teamcity
Tech fuse11 toolingtestingci-vs2010teamcityTech fuse11 toolingtestingci-vs2010teamcity
Tech fuse11 toolingtestingci-vs2010teamcity
 
Data Science: Good, Bad and Ugly by Irina Kukuyeva
Data Science: Good, Bad and Ugly by Irina KukuyevaData Science: Good, Bad and Ugly by Irina Kukuyeva
Data Science: Good, Bad and Ugly by Irina Kukuyeva
 
How I ended up touching Magento core
How I ended up touching Magento coreHow I ended up touching Magento core
How I ended up touching Magento core
 
Fri benghiat gil-odsc-data-kitchen-data science to dataops
Fri benghiat gil-odsc-data-kitchen-data science to dataopsFri benghiat gil-odsc-data-kitchen-data science to dataops
Fri benghiat gil-odsc-data-kitchen-data science to dataops
 
ODSC data science to DataOps
ODSC data science to DataOpsODSC data science to DataOps
ODSC data science to DataOps
 
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"
JDD 2016 - Joseph W. Yoder - Deliver Fast "With Confidence"
 
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology OrgsWhy Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
 
Continuous delivery agile_2012
Continuous delivery agile_2012Continuous delivery agile_2012
Continuous delivery agile_2012
 
Sencha Touch 2
Sencha Touch 2Sencha Touch 2
Sencha Touch 2
 

Mais de Kellan

More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.Kellan
 
Optimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceOptimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceKellan
 
More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.Kellan
 
Future of handmade
Future of handmadeFuture of handmade
Future of handmadeKellan
 
Architecting for Change: QCONNYC 2012
Architecting for Change: QCONNYC 2012Architecting for Change: QCONNYC 2012
Architecting for Change: QCONNYC 2012Kellan
 
Metrics driven engineering (velocity 2011)
Metrics driven engineering (velocity 2011)Metrics driven engineering (velocity 2011)
Metrics driven engineering (velocity 2011)Kellan
 
Solving the "Brooklyn Problem"
Solving the "Brooklyn Problem" Solving the "Brooklyn Problem"
Solving the "Brooklyn Problem" Kellan
 
Social Software For Robots
Social Software For RobotsSocial Software For Robots
Social Software For RobotsKellan
 
Beyond REST? Building data services with XMPP
Beyond REST? Building data services with XMPPBeyond REST? Building data services with XMPP
Beyond REST? Building data services with XMPPKellan
 
Advanced OAuth Wrangling
Advanced OAuth WranglingAdvanced OAuth Wrangling
Advanced OAuth WranglingKellan
 
Casual Privacy (Ignite Web2.0 Expo)
Casual Privacy (Ignite Web2.0 Expo)Casual Privacy (Ignite Web2.0 Expo)
Casual Privacy (Ignite Web2.0 Expo)Kellan
 

Mais de Kellan (11)

More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.
 
Optimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerceOptimizing for change: Taking risks safely & e-commerce
Optimizing for change: Taking risks safely & e-commerce
 
More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.More women in engineering: Something that ACTUALLY WORKED.
More women in engineering: Something that ACTUALLY WORKED.
 
Future of handmade
Future of handmadeFuture of handmade
Future of handmade
 
Architecting for Change: QCONNYC 2012
Architecting for Change: QCONNYC 2012Architecting for Change: QCONNYC 2012
Architecting for Change: QCONNYC 2012
 
Metrics driven engineering (velocity 2011)
Metrics driven engineering (velocity 2011)Metrics driven engineering (velocity 2011)
Metrics driven engineering (velocity 2011)
 
Solving the "Brooklyn Problem"
Solving the "Brooklyn Problem" Solving the "Brooklyn Problem"
Solving the "Brooklyn Problem"
 
Social Software For Robots
Social Software For RobotsSocial Software For Robots
Social Software For Robots
 
Beyond REST? Building data services with XMPP
Beyond REST? Building data services with XMPPBeyond REST? Building data services with XMPP
Beyond REST? Building data services with XMPP
 
Advanced OAuth Wrangling
Advanced OAuth WranglingAdvanced OAuth Wrangling
Advanced OAuth Wrangling
 
Casual Privacy (Ignite Web2.0 Expo)
Casual Privacy (Ignite Web2.0 Expo)Casual Privacy (Ignite Web2.0 Expo)
Casual Privacy (Ignite Web2.0 Expo)
 

Último

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 

Último (20)

Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 

Engineering Change

  • 1. Engineering Change: Enable rapid experimentation Kellan Elliott-McCrea @kellan CTO, Etsy Thursday, May 17, 12
  • 2. * A marketplace for handmade and vintage goods. In April 2012 * 875,000 artisans and creative independents * $63.6 million USD in sales * 1.32 billion page views Thursday, May 17, 12
  • 3. Engineering Change: * Continuous Deployment * Metrics driven development Thursday, May 17, 12
  • 4. Engineering Change: * Make mistakes quickly * Learn from mistakes Thursday, May 17, 12
  • 5. Engineering Change: * Continuous Deployment * Metrics driven development Thursday, May 17, 12
  • 6. Continuous Deployment: a technique for rapid iteration Thursday, May 17, 12
  • 7. Continuous Deployment: make failure cheap Thursday, May 17, 12
  • 8. Continuous Deployment: small changes, pushed frequently Thursday, May 17, 12
  • 9. Continuous Deployment: small changes, pushed frequently. To production. Thursday, May 17, 12
  • 10. Continuous Deployment: 10,068 releases to production in 2011 @ Etsy Thursday, May 17, 12
  • 11. Continuous Deployment: How? Thursday, May 17, 12
  • 12. Continuous Deployment: No branching. “All existing revision control systems were built by people who build installed software” - Paul Hammond, Always Ship Trunk, Velocity 2010 Thursday, March 17, 2011 Thursday, May 17, 12
  • 13. Continuous Deployment: feature flags if ($cfg[‘awesome_new_search’]) { # new hotness $rsp = do_solr(); } else { # boring old stuff $rsp = do_grep(); } Thursday, May 17, 12
  • 14. Continuous Deployment: Ramp - ups (on top of feature flags) 1. Launch to staff only 2. Launch to 1% of all users 3. Launch to members of a beta group Thursday, May 17, 12
  • 15. Continuous Deployment: any engineer can launch a feature to 1% of users Thursday, May 17, 12
  • 16. Continuous Deployment: ~200 experiments live right now Thursday, May 17, 12
  • 17. Metrics driven development: optimize for learning Thursday, May 17, 12
  • 18. Metrics driven development: measure everything Thursday, May 17, 12
  • 19. Metrics driven development: getting started: make it easy Thursday, May 17, 12
  • 20. Metrics driven development: measure everything: logs, monitoring, A/B, graphs, StatsD Thursday, May 17, 12
  • 21. Metrics driven development: StatsD @ Instagram, Pinterest, Github, Mozilla, LAN.com, Zynga, Kickstarter, LivingSocial and 70+ other companies Thursday, May 17, 12
  • 22. Metrics driven development: StatsD “We love statsd at Instagram. Written by Etsy, it’s a network daemon that aggregates and rolls-up data into Graphite. At its core, it has two types of statistics: counter and timers. We use the counters to track everything from number of signups per second to number of likes, and we use timers to time generation of feeds, how long it takes to follow users, and any other major action.” - Mike Krieger, Founder and CTO, Instagram. Thursday, May 17, 12
  • 23. Metrics driven development: StatsD Open source, a daemon for timing and counting things, graphs rates, averages and 95th percentile, 20-30 minute install, easy to use Thursday, May 17, 12
  • 24. Metrics driven development StatsD::timing("page.render", $msec); Thursday, May 17, 12
  • 26. Metrics driven development getting started: your 5 core metrics Thursday, May 17, 12
  • 27. Metrics driven development 5 core metrics @ Etsy: sign ups, logins, checkout, new listings, posts in the bugs forums Thursday, May 17, 12
  • 28. Metrics driven development 378,000 metrics, collected roughly once a second Thursday, May 17, 12
  • 29. Metrics driven development the confidence to change Thursday, May 17, 12
  • 30. Engineering change: Rapid experimentation Adapt, find market fit. Thursday, May 17, 12
  • 31. Engineering change: Rapid experimentation Adapt, find market fit, addictive to engineers. Thursday, May 17, 12
  • 32. Thank you! Lean more: http://codeascraft.etsy.com Thursday, May 17, 12