SlideShare uma empresa Scribd logo

Mozilla: Continuous Deploment on SUMO

•
•
Matt Brandt
Matt Brandt

An overview of how Mozilla is doing Continuous Deployment for support.mozilla.org

TecnologiaNotícias e política
1 de 49
Baixar para ler offline
MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
[Nearly] MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
Bring questions Tuesday, November 22, 2011
Bring questions Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? Tuesday, November 22, 2011
Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? How do we measure quality? Tuesday, November 22, 2011
But first a little history... Tuesday, November 22, 2011
But first a little history... Tuesday, November 22, 2011
Short Iterations Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •IT pushes •Is this the right thing? •Regression test •Verify •Automate •Exploratory Testing •Wahoo! •Is this the right thing? Tuesday, November 22, 2011
The brave new world Continuous deployment http://cliptank.com/funny/dog-costumes.php Tuesday, November 22, 2011
Tools & attitude Tuesday, November 22, 2011
Discuss what is acceptable risk https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. • Example: the ability to answer a question. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
Acceptable risk Tuesday, November 22, 2011
Acceptable risk Tuesday, November 22, 2011
What’s our manual strategy? Tuesday, November 22, 2011
What’s our manual strategy? Tuesday, November 22, 2011
Manual strategy Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Testing can occur in production Tuesday, November 22, 2011
Manual strategy Feature verification & exploratory testing Testing can occur in production Community involvement on new features Tuesday, November 22, 2011
Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
Automation strategy Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work Tuesday, November 22, 2011
Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work [All] Tests will be deleted if they don’t cover critical areas Tuesday, November 22, 2011
Waffling Failure is always an option Tuesday, November 22, 2011
Waffling Failure is always an option Tuesday, November 22, 2011
Control Tuesday, November 22, 2011
Gathering data Tuesday, November 22, 2011
Gathering data Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Automation runs •Fast iterations •Early stage testing Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Automation runs •Auto-updated •Fast iterations •No post-commit hooks •Early stage testing •Bug & feature testing Tuesday, November 22, 2011
Now Trunk/ Stage Prod Dev •Not push-button •Automation runs •Auto-updated •Automation •Fast iterations •No post-commit hooks •Monitoring •Early stage testing •Bug & feature testing •Feedback tool •Wahoo! Tuesday, November 22, 2011
Questions Tuesday, November 22, 2011
References •https://support.mozilla.com •https://wiki.mozilla.org/QA/Execution/Web_Testing/ •http://docs.jquery.com/QUnit Continuous_Deployment •https://github.com/mozilla/sumo-tests •https://github.com/jsocol/django-waffle •http://coffeeonthekeyboard.com/the-future-of-sumo- •http://coffeeonthekeyboard.com/introducing-waffle-for- django-541/ development-511/ •http://coffeeonthekeyboard.com/acronyms-you-should-know- •https://github.com/etsy/statsd mttd-and-mttr-597/ •http://codeascraft.etsy.com/2011/02/15/measure-anything- measure-everything/ •http://www.slideshare.net/stephendonner/continuous- deployment-9363895 Tuesday, November 22, 2011

Recomendados

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps WorldKit Plummer
 
Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Atlassian
 
Alternative Software Development Methodology
Alternative Software Development MethodologyAlternative Software Development Methodology
Alternative Software Development Methodologyทวิร พานิชสมบัติ
 
Test Driven Sysadmin
Test Driven SysadminTest Driven Sysadmin
Test Driven Sysadminjohanku
 
obs-tdd-intro
obs-tdd-introobs-tdd-intro
obs-tdd-introAmir Barylko
 
3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time 3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time Pascal Rettig
 
Agile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamAgile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamdrewz lin
 
Monitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationMonitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationTheo Schlossnagle
 

Recomendados

Quality Assurance in a DevOps World
Quality Assurance in a DevOps WorldQuality Assurance in a DevOps World
Quality Assurance in a DevOps WorldKit Plummer
 
Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Bonfire... How'd You Do That?! - AtlasCamp 2011
Bonfire... How'd You Do That?! - AtlasCamp 2011Atlassian
 
Alternative Software Development Methodology
Alternative Software Development MethodologyAlternative Software Development Methodology
Alternative Software Development Methodologyทวิร พานิชสมบัติ
 
Test Driven Sysadmin
Test Driven SysadminTest Driven Sysadmin
Test Driven Sysadminjohanku
 
obs-tdd-intro
obs-tdd-introobs-tdd-intro
obs-tdd-introAmir Barylko
 
3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time 3D in the Browser via WebGL: It's Go Time
3D in the Browser via WebGL: It's Go Time Pascal Rettig
 
Agile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamAgile2012 soccer witha_basketballteam
Agile2012 soccer witha_basketballteamdrewz lin
 
Monitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationMonitoring is easy, why are we so bad at it presentation
Monitoring is easy, why are we so bad at it presentationTheo Schlossnagle
 
Donating a mature project to Eclipse
Donating a mature project to EclipseDonating a mature project to Eclipse
Donating a mature project to Eclipseglynnormington
 
ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut PotsdamMichael Palotas
 
Agile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureAgile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureWee Witthawaskul
 
Building Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileBuilding Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileWee Witthawaskul
 
Get your Project back in Shape!
Get your Project back in Shape!Get your Project back in Shape!
Get your Project back in Shape!Joachim Tuchel
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for youAmbientia
 
Agile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveAgile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveWee Witthawaskul
 
Asteroïden
AsteroïdenAsteroïden
AsteroïdenThomasBos
 
Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1teoper1205
 
Appnostic
AppnosticAppnostic
AppnosticVeronika Kormanyos
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 aprVeronika Kormanyos
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2twi8er
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 aprVeronika Kormanyos
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 aprVeronika Kormanyos
 
Wifi work of july 19, paris
Wifi work of july 19, parisWifi work of july 19, paris
Wifi work of july 19, parisVeronika Kormanyos
 
Dreamitalive
DreamitaliveDreamitalive
DreamitaliveVeronika Kormanyos
 
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...bradyjay
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalRenato Sfolia
 
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisDOGSCAMP Summit
 

Mais conteúdo relacionado

Mais procurados

Donating a mature project to Eclipse
Donating a mature project to EclipseDonating a mature project to Eclipse
Donating a mature project to Eclipseglynnormington
 
ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut PotsdamMichael Palotas
 
Agile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureAgile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureWee Witthawaskul
 
Building Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileBuilding Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileWee Witthawaskul
 
Get your Project back in Shape!
Get your Project back in Shape!Get your Project back in Shape!
Get your Project back in Shape!Joachim Tuchel
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for youAmbientia
 
Agile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveAgile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveWee Witthawaskul
 

Mais procurados (7)

Donating a mature project to Eclipse
Donating a mature project to EclipseDonating a mature project to Eclipse
Donating a mature project to Eclipse
 
ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdam
 
Agile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai CultureAgile Software Development Process Practice in Thai Culture
Agile Software Development Process Practice in Thai Culture
 
Building Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed AgileBuilding Mobile (app) Masterpiece with Distributed Agile
Building Mobile (app) Masterpiece with Distributed Agile
 
Get your Project back in Shape!
Get your Project back in Shape!Get your Project back in Shape!
Get your Project back in Shape!
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
 
Agile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer PerspectiveAgile Software Development in Practice - A Developer Perspective
Agile Software Development in Practice - A Developer Perspective
 

Destaque

Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1teoper1205
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2twi8er
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortVeronika Kormanyos
 
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...bradyjay
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalRenato Sfolia
 

Destaque (14)

Asteroïden
AsteroïdenAsteroïden
Asteroïden
 
Webprog lecture1
Webprog lecture1Webprog lecture1
Webprog lecture1
 
Appnostic
AppnosticAppnostic
Appnostic
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Presentation zahid 2
Presentation zahid 2Presentation zahid 2
Presentation zahid 2
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Rtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effortRtz Carlton and Jet Blue app. effort
Rtz Carlton and Jet Blue app. effort
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Think tank – paris 27 apr
Think tank – paris 27 aprThink tank – paris 27 apr
Think tank – paris 27 apr
 
Wifi work of july 19, paris
Wifi work of july 19, parisWifi work of july 19, paris
Wifi work of july 19, paris
 
Dreamitalive
DreamitaliveDreamitalive
Dreamitalive
 
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
Scrolling Beyond Binaries: Exploring social media use amongst young LGBTIQ+ ...
 
PCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde OcupacionalPCMSO - Programa de Controle Médico de Saúde Ocupacional
PCMSO - Programa de Controle Médico de Saúde Ocupacional
 

Semelhante a Mozilla: Continuous Deploment on SUMO

ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisDOGSCAMP Summit
 
Agile Deployments with Puppet
Agile Deployments with PuppetAgile Deployments with Puppet
Agile Deployments with PuppetPuppet
 
PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?Andrew Mleczko
 
Pitfalls of Continuous Deployment
Pitfalls of Continuous DeploymentPitfalls of Continuous Deployment
Pitfalls of Continuous Deploymentzeeg
 
Ruby can haz Everyone
Ruby can haz EveryoneRuby can haz Everyone
Ruby can haz EveryoneRubyX
 
Building an experimentation framework
Building an experimentation frameworkBuilding an experimentation framework
Building an experimentation frameworkzsqr
 
Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)zeeg
 
Gradle - The hackable build-tool
Gradle - The hackable build-toolGradle - The hackable build-tool
Gradle - The hackable build-toolLeonard Axelsson
 
Recognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget SoundRecognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget SoundChris Sterling
 
every-day-automation
every-day-automationevery-day-automation
every-day-automationAmir Barylko
 
JavaSE - The road forward
JavaSE - The road forwardJavaSE - The road forward
JavaSE - The road forwardeug3n_cojocaru
 
SecurityBSides las vegas - Agnitio
SecurityBSides las vegas - AgnitioSecurityBSides las vegas - Agnitio
SecurityBSides las vegas - AgnitioSecurity Ninja
 
Android 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journeyAndroid 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journeyEmanuele Di Saverio
 
Is Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic neededIs Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic neededchiportal
 
Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011) Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011) Leonardo Borges
 
Plugin Testing
Plugin TestingPlugin Testing
Plugin TestingTim Moore
 
Integrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio ManagementIntegrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio ManagementChris Sterling
 

Semelhante a Mozilla: Continuous Deploment on SUMO (20)

ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias SociaisExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
ExpOn 2011 - Diego Monteiro - Níveis de Maturidade nas Mídias Sociais
 
Agile Deployments with Puppet
Agile Deployments with PuppetAgile Deployments with Puppet
Agile Deployments with Puppet
 
PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?PyconUA - How to build ERP application having fun?
PyconUA - How to build ERP application having fun?
 
Pitfalls of Continuous Deployment
Pitfalls of Continuous DeploymentPitfalls of Continuous Deployment
Pitfalls of Continuous Deployment
 
Agile xptdd@gosoft
Agile xptdd@gosoftAgile xptdd@gosoft
Agile xptdd@gosoft
 
Agile xp tdd@gosoft
Agile xp tdd@gosoftAgile xp tdd@gosoft
Agile xp tdd@gosoft
 
Ruby can haz Everyone
Ruby can haz EveryoneRuby can haz Everyone
Ruby can haz Everyone
 
Hudson
HudsonHudson
Hudson
 
Building an experimentation framework
Building an experimentation frameworkBuilding an experimentation framework
Building an experimentation framework
 
Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)Continuous Deployment at Disqus (Pylons Minicon)
Continuous Deployment at Disqus (Pylons Minicon)
 
Gradle - The hackable build-tool
Gradle - The hackable build-toolGradle - The hackable build-tool
Gradle - The hackable build-tool
 
Recognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget SoundRecognizing Software Debt - Beyond Agile Puget Sound
Recognizing Software Debt - Beyond Agile Puget Sound
 
every-day-automation
every-day-automationevery-day-automation
every-day-automation
 
JavaSE - The road forward
JavaSE - The road forwardJavaSE - The road forward
JavaSE - The road forward
 
SecurityBSides las vegas - Agnitio
SecurityBSides las vegas - AgnitioSecurityBSides las vegas - Agnitio
SecurityBSides las vegas - Agnitio
 
Android 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journeyAndroid 1.5 to 3.0: a compatibility journey
Android 1.5 to 3.0: a compatibility journey
 
Is Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic neededIs Advanced Verification for FPGA based Logic needed
Is Advanced Verification for FPGA based Logic needed
 
Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011) Clouds against the Floods (RubyConfBR2011)
Clouds against the Floods (RubyConfBR2011)
 
Plugin Testing
Plugin TestingPlugin Testing
Plugin Testing
 
Integrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio ManagementIntegrating Quality into Project Portfolio Management
Integrating Quality into Project Portfolio Management
 

Último

Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 

Último (20)

Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 

Mozilla: Continuous Deploment on SUMO

  • 1. MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
  • 2. [Nearly] MOZILLA Continuous Deployment on Sumo support.mozilla.com Matt Brandt, QA Engineer mbrandt@mozilla.com Tuesday, November 22, 2011
  • 5. Bring questions What’s wrong with our current small iterations? Tuesday, November 22, 2011
  • 6. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! Tuesday, November 22, 2011
  • 7. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? Tuesday, November 22, 2011
  • 8. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? Tuesday, November 22, 2011
  • 9. Bring questions What’s wrong with our current small iterations? Trust developers to test the app??!! What happens to our Selenium test suite? We’ll be relegated to manual testers? How do we measure quality? Tuesday, November 22, 2011
  • 10. But first a little history... Tuesday, November 22, 2011
  • 11. But first a little history... Tuesday, November 22, 2011
  • 13. Simplified What the team did before Trunk/ Stage Prod Dev Tuesday, November 22, 2011
  • 14. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
  • 15. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •Is this the right thing? •Automate Tuesday, November 22, 2011
  • 16. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
  • 17. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •Is this the right thing? •Regression test •Automate •Exploratory Testing •Is this the right thing? Tuesday, November 22, 2011
  • 18. Simplified What the team did before Trunk/ Stage Prod Dev •Verify new features •IT pushes •IT pushes •Is this the right thing? •Regression test •Verify •Automate •Exploratory Testing •Wahoo! •Is this the right thing? Tuesday, November 22, 2011
  • 19. The brave new world Continuous deployment http://cliptank.com/funny/dog-costumes.php Tuesday, November 22, 2011
  • 20. Tools & attitude Tuesday, November 22, 2011
  • 21. Discuss what is acceptable risk https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 22. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 23. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 24. Discuss what is acceptable risk Risk: Less critical areas will not be covered by automation •If the feature breaks, it is acceptable to wait up to an hour for a fix. • Example: the ability to answer a question. https://wiki.mozilla.org/QA/Execution/Web_Testing/Continuous_Deployment#Risks_.26_Plans_.26_Tools Tuesday, November 22, 2011
  • 27. What’s our manual strategy? Tuesday, November 22, 2011
  • 28. What’s our manual strategy? Tuesday, November 22, 2011
  • 30. Manual strategy Feature verification & exploratory testing Tuesday, November 22, 2011
  • 31. Manual strategy Feature verification & exploratory testing Testing can occur in production Tuesday, November 22, 2011
  • 32. Manual strategy Feature verification & exploratory testing Testing can occur in production Community involvement on new features Tuesday, November 22, 2011
  • 33. Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
  • 34. Automation is fun https://github.com/mozilla/sumo-tests Tuesday, November 22, 2011
  • 36. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Tuesday, November 22, 2011
  • 37. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Tuesday, November 22, 2011
  • 38. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work Tuesday, November 22, 2011
  • 39. Automation strategy Selenium tests for areas that are clumsy to cover with unit tests or QUnit Qualify as deployment blockers: run in dev & staging envs Non-volatile production tests verify env & services work [All] Tests will be deleted if they don’t cover critical areas Tuesday, November 22, 2011
  • 40. Waffling Failure is always an option Tuesday, November 22, 2011
  • 41. Waffling Failure is always an option Tuesday, November 22, 2011
  • 45. Now Trunk/ Stage Prod Dev •Automation runs •Fast iterations •Early stage testing Tuesday, November 22, 2011
  • 46. Now Trunk/ Stage Prod Dev •Automation runs •Auto-updated •Fast iterations •No post-commit hooks •Early stage testing •Bug & feature testing Tuesday, November 22, 2011
  • 47. Now Trunk/ Stage Prod Dev •Not push-button •Automation runs •Auto-updated •Automation •Fast iterations •No post-commit hooks •Monitoring •Early stage testing •Bug & feature testing •Feedback tool •Wahoo! Tuesday, November 22, 2011
  • 49. References •https://support.mozilla.com •https://wiki.mozilla.org/QA/Execution/Web_Testing/ •http://docs.jquery.com/QUnit Continuous_Deployment •https://github.com/mozilla/sumo-tests •https://github.com/jsocol/django-waffle •http://coffeeonthekeyboard.com/the-future-of-sumo- •http://coffeeonthekeyboard.com/introducing-waffle-for- django-541/ development-511/ •http://coffeeonthekeyboard.com/acronyms-you-should-know- •https://github.com/etsy/statsd mttd-and-mttr-597/ •http://codeascraft.etsy.com/2011/02/15/measure-anything- measure-everything/ •http://www.slideshare.net/stephendonner/continuous- deployment-9363895 Tuesday, November 22, 2011