O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Cloud Developer’s DHARMA… 
redefining ‘done’ for Cloud applications 
Daniel Bryant 
Principal Consultant, Open Credo 
dani...
What to expect 
• Problems when developing for the Cloud 
– “Lift and shift” 
– Smashing the monolith 
– Greenfield 
• Som...
Who Am I? 
• Principal Consultant at Open Credo 
– Agile transformations 
– DevOps 
– Microservices and Cloud 
• LJC Assoc...
Core Changes… 
• Service-Oriented Architecture 
• Cloud-based deployments 
• DevOps Culture 
01/10/2014 @danielbryantuk
Core Changes… 
• Service-Oriented Architecture 
– Twitter’s Story (bit.ly/1j1WbmI) 
• Cloud-based deployments 
– Today! 
•...
Common Cloud Problems 
TL;DR… 
01/10/2014 @danielbryantuk
Not respecting the underlying environment 
01/10/2014 @danielbryantuk
Lack of application/platform monitoring… 
01/10/2014 @danielbryantuk
Bizarre failure modes… 
01/10/2014 @danielbryantuk
Difficult to understand 
01/10/2014 @danielbryantuk 
the architecture
Not testing in the Cloud… 
(hint: here be dragons!) 
01/10/2014 @danielbryantuk
We’ve created a “Cloud Developer’s DHARMA” 
to act as a checklist when building Cloud apps 
01/10/2014 @danielbryantuk
dharma 
/ˈdɑːmə,ˈdəːmə/ 
noun 
1. Signifies behaviors that are considered to be in 
accord with order that makes life and ...
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
01/10/2014 @danielbryantuk
Documentation (just enough) 
• Provide a map for developers 
• Component purpose (and contract) 
• Initialisation instruct...
Simon Brown’s C4 Model 
http://www.codingthearchitecture.com/ 
01/10/2014 @danielbryantuk
01/10/2014 @danielbryantuk
0011//1100//22010414 @@ddaanniieellbbrryyaanntutukk
API Docs with Swagger 
https://helloreverb.com/developers/swagger 
01/10/2014 @danielbryantuk
Create a PACT 
01/10/2014 @danielbryantuk 
https://github.com/DiUS/pact-jvm
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
High Cohesion / Loose Coupling 
(all the way down…) 
• Code 
• Architecture 
– Components 
– Services 
• Public API 
– Pay...
Smashing the Monolith… 
• Business functionality – “Cart Service” 
• Technology chunk - “Email Service” 
• Vertical Slice ...
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
Automated from Commit to Cloud 
• Continuous Integration 
• Continuous Deployment 
• Continuous Delivery 
01/10/2014 @dani...
Our Build Pipeline 
Jenkins, with plugins… 
• Build Pipeline 
– wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin ...
Our Build Pipeline 
• Component Build 
– Compile 
– Unit Tests (surefire) 
– Integration Tests (failsafe) 
• Deployment on...
Our Build Pipeline 
• Acceptance Tests 
– Cucumber (and Selenium) 
• Performance Tests 
– Jmeter + Jenkins performance plu...
Automating QA 
• Intra-component integration testing 
– Utilise embedded datastore/middleware 
– “Scassandra” (github.com/...
Infrastructure: Say No To Snowflakes! 
• Automate all provisioning (store in SCM) 
• Fry... 
– Chef, Puppet, SaltStack, An...
Infrastructure: Say No To Snowflakes! 
• Doing “Proper Development” 
– Gareth Rushgrove at Craft Conf (bit.ly/1njuc49) 
– ...
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
Deployment Platform: What you’ve got… 
01/10/2014 @danielbryantuk
What you think you want… 
01/10/2014 @danielbryantuk
What you get… 
Fact: 9 out of 10 cheetahs prefer the 
taste of an Ops team over tinned food 
01/10/2014 @danielbryantuk
Thou Shalt Know thy Cloud… 
• AWS “Magnetic” EBS 100 IOPS (by default) 
– My Mac SSD does 49K IOPS 
• 1000Mbps network max...
Thinking/Acting Operationally 
• Cultivate “Mechanical Sympathy” 
• Virtualisation 
– Tech Target (bit.ly/1kDVqyG) 
• Netw...
Thinking/Acting Operationally 
• Learn Linux fundamentals 
• Diagnostic skills 
– top, netstat, vmstat, tcpdump 
– Java ut...
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
Monitor All The Things! 
• Infrastructure monitoring 
– Nagios / Zabbix 
– AppDynamics 
• Centralised Logging 
• Distribut...
Component Metrics 
• Dropwizard’s Metrics 
– metrics.codahale.com 
• Netflix’s Servo 
– github.com/Netflix/servo 
• Etsy’s...
Health Checks 
01/10/2014 @danielbryantuk
Gauges, Counters, Meters, Timers… 
01/10/2014 @danielbryantuk
Graph It! 
01/10/2014 @danielbryantuk
01/10/2014 @danielbryantuk
01/10/2014 @danielbryantuk
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated from commit to Cloud 
Resource awa...
Antifragile 
• The opposite of fragile? 
– Robust… 
– Antifragile… 
• Netflix are best-in-class 
– bit.ly/1gs5n3q 
• Syste...
Design for Failure 
• Design patterns 
– Timeouts / retries 
– Bulkheads / circuit-breakers 
• Inspiration 
– Chris Richar...
Retries 
https://github.com/rholder/guava-retrying 
01/10/2014 @danielbryantuk
Circuit-breaker 
01/10/2014 @danielbryantuk 
https://github.com/Netflix/Hystrix 
https://github.com/Netflix/Hystrix/tree/m...
Robust in the Cloud 
• Distributed Computing Principles 
– ‘For young bloods’ (bit.ly/1pKVepz) 
• Check your business goal...
Antifragile Patterns: Respect the CAP 
Eventual consistency (ACID vs BASE) 
Clever caching (soft-state) 
http://en.wikiped...
Antifragile Patterns: Elastic Scaling 
Stateless components 
Distributed data stores / caches 
Asynchronous communication ...
So, Cloud Apps are ‘done’ when… 
Documented (just enough) 
Highly cohesive/loosely coupled (all the way down) 
Automated f...
Thanks For Listening 
• Massive thanks to all the Open Credo team! 
• Questions / comments? 
– daniel.bryant@opencredo.com...
Próximos SlideShares
Carregando em…5
×

JavaOne 2014: Cloud Developer's DHARMA: Redefining 'done' for Cloud applications

4.284 visualizações

Publicada em

Building Java applications for the IaaS cloud is easy, right? “Sure, no problem. Just lift and shift,” all the cloud vendors shout in unison. However, the reality of building and deploying cloud applications can often be different. This session introduces lessons learned from the trenches during several years of designing and implementing cloud-based Java applications, which we have codified into our Cloud Developer's “DHARMA” rules: Documented (just enough); Highly cohesive/loosely coupled (all the way down); Automated from code commit to cloud; Resource-aware; Monitored thoroughly; and Antifragile.

This session was presented at JavaOne 2014

Publicada em: Tecnologia
  • Seja o primeiro a comentar

JavaOne 2014: Cloud Developer's DHARMA: Redefining 'done' for Cloud applications

  1. 1. Cloud Developer’s DHARMA… redefining ‘done’ for Cloud applications Daniel Bryant Principal Consultant, Open Credo daniel.bryant@opencredo.com @danielbryantuk
  2. 2. What to expect • Problems when developing for the Cloud – “Lift and shift” – Smashing the monolith – Greenfield • Some suggestions on where to focus efforts • Tools and techniques • Lots of information… (slides will be available) 01/10/2014 @danielbryantuk
  3. 3. Who Am I? • Principal Consultant at Open Credo – Agile transformations – DevOps – Microservices and Cloud • LJC Associate • Adopt OpenJDK and JSR 01/10/2014 @danielbryantuk
  4. 4. Core Changes… • Service-Oriented Architecture • Cloud-based deployments • DevOps Culture 01/10/2014 @danielbryantuk
  5. 5. Core Changes… • Service-Oriented Architecture – Twitter’s Story (bit.ly/1j1WbmI) • Cloud-based deployments – Today! • DevOps Culture – Devoxx UK talk (bit.ly/1BylnZb) – Previous LJC Event (bit.ly/1elVPJz) 01/10/2014 @danielbryantuk
  6. 6. Common Cloud Problems TL;DR… 01/10/2014 @danielbryantuk
  7. 7. Not respecting the underlying environment 01/10/2014 @danielbryantuk
  8. 8. Lack of application/platform monitoring… 01/10/2014 @danielbryantuk
  9. 9. Bizarre failure modes… 01/10/2014 @danielbryantuk
  10. 10. Difficult to understand 01/10/2014 @danielbryantuk the architecture
  11. 11. Not testing in the Cloud… (hint: here be dragons!) 01/10/2014 @danielbryantuk
  12. 12. We’ve created a “Cloud Developer’s DHARMA” to act as a checklist when building Cloud apps 01/10/2014 @danielbryantuk
  13. 13. dharma /ˈdɑːmə,ˈdəːmə/ noun 1. Signifies behaviors that are considered to be in accord with order that makes life and universe possible (Hinduism) 2. "cosmic law and order”, but is also applied to the teachings of the Buddha (Buddhism) 01/10/2014 @danielbryantuk
  14. 14. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  15. 15. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  16. 16. 01/10/2014 @danielbryantuk
  17. 17. Documentation (just enough) • Provide a map for developers • Component purpose (and contract) • Initialisation instructions • Highlights areas of operational risk 01/10/2014 @danielbryantuk
  18. 18. Simon Brown’s C4 Model http://www.codingthearchitecture.com/ 01/10/2014 @danielbryantuk
  19. 19. 01/10/2014 @danielbryantuk
  20. 20. 0011//1100//22010414 @@ddaanniieellbbrryyaanntutukk
  21. 21. API Docs with Swagger https://helloreverb.com/developers/swagger 01/10/2014 @danielbryantuk
  22. 22. Create a PACT 01/10/2014 @danielbryantuk https://github.com/DiUS/pact-jvm
  23. 23. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  24. 24. High Cohesion / Loose Coupling (all the way down…) • Code • Architecture – Components – Services • Public API – PayPal (bit.ly/1hnZNly) 01/10/2014 @danielbryantuk
  25. 25. Smashing the Monolith… • Business functionality – “Cart Service” • Technology chunk - “Email Service” • Vertical Slice – “Registration” (Groupon: vimeo.com/105880150) • Horizontal Slice – “User Repo” (Microservices: oreil.ly/1pp6qmx) 01/10/2014 @danielbryantuk
  26. 26. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  27. 27. Automated from Commit to Cloud • Continuous Integration • Continuous Deployment • Continuous Delivery 01/10/2014 @danielbryantuk
  28. 28. Our Build Pipeline Jenkins, with plugins… • Build Pipeline – wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin • Parameterized build – wiki.jenkins-ci.org/display/JENKINS/Parameterized+Build • Promoted Builds Plugin – wiki.jenkins-ci.org/display/JENKINS/Promoted+Builds+Plugin 01/10/2014 @danielbryantuk
  29. 29. Our Build Pipeline • Component Build – Compile – Unit Tests (surefire) – Integration Tests (failsafe) • Deployment onto QA Cloud – Python Scripts + Chef to provision – Verify success using Python 01/10/2014 @danielbryantuk
  30. 30. Our Build Pipeline • Acceptance Tests – Cucumber (and Selenium) • Performance Tests – Jmeter + Jenkins performance plugin – Make sure environment is realistic!! • Live Deployment? 01/10/2014 @danielbryantuk
  31. 31. Automating QA • Intra-component integration testing – Utilise embedded datastore/middleware – “Scassandra” (github.com/scassandra) – Service virtualisation (www.mbtest.org) • Inter-component integration testing – The hardest part of SOA… – Consider ‘synthetic txns’ 01/10/2014 @danielbryantuk
  32. 32. Infrastructure: Say No To Snowflakes! • Automate all provisioning (store in SCM) • Fry... – Chef, Puppet, SaltStack, Ansible – Bash, Python (Fabric) – Vendor APIs • …or bake? – Packer.io – Netflix Aminator 01/10/2014 @danielbryantuk
  33. 33. Infrastructure: Say No To Snowflakes! • Doing “Proper Development” – Gareth Rushgrove at Craft Conf (bit.ly/1njuc49) – Chef Conf (www.youtube.com/user/getchef) • Local tooling/testing – Vagrant (www.vagrantup.com) – Docker (www.docker.io) 01/10/2014 @danielbryantuk
  34. 34. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  35. 35. Deployment Platform: What you’ve got… 01/10/2014 @danielbryantuk
  36. 36. What you think you want… 01/10/2014 @danielbryantuk
  37. 37. What you get… Fact: 9 out of 10 cheetahs prefer the taste of an Ops team over tinned food 01/10/2014 @danielbryantuk
  38. 38. Thou Shalt Know thy Cloud… • AWS “Magnetic” EBS 100 IOPS (by default) – My Mac SSD does 49K IOPS • 1000Mbps network max transfer ~125MB/s – My Mac does 400+ MB/s Sequential Write to SSD • “Noisy [virtual] Neighbours” Reference for Mac statistics: bit.ly/1ftJZH8 01/10/2014 @danielbryantuk
  39. 39. Thinking/Acting Operationally • Cultivate “Mechanical Sympathy” • Virtualisation – Tech Target (bit.ly/1kDVqyG) • Networking – ‘Unix and Linux System Administration Handbook’ – aws.amazon.com/documentation 01/10/2014 @danielbryantuk
  40. 40. Thinking/Acting Operationally • Learn Linux fundamentals • Diagnostic skills – top, netstat, vmstat, tcpdump – Java utils: jps, jstat, jmap, jhat – “DevOps Troubleshooting” by K. Rankin • Maybe grow a beard… 01/10/2014 @danielbryantuk
  41. 41. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  42. 42. Monitor All The Things! • Infrastructure monitoring – Nagios / Zabbix – AppDynamics • Centralised Logging • Distributed Tracing – twitter.github.io/zipkin 01/10/2014 @danielbryantuk
  43. 43. Component Metrics • Dropwizard’s Metrics – metrics.codahale.com • Netflix’s Servo – github.com/Netflix/servo • Etsy’s StatsD – github.com/etsy/statsd/wiki 01/10/2014 @danielbryantuk
  44. 44. Health Checks 01/10/2014 @danielbryantuk
  45. 45. Gauges, Counters, Meters, Timers… 01/10/2014 @danielbryantuk
  46. 46. Graph It! 01/10/2014 @danielbryantuk
  47. 47. 01/10/2014 @danielbryantuk
  48. 48. 01/10/2014 @danielbryantuk
  49. 49. Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  50. 50. Antifragile • The opposite of fragile? – Robust… – Antifragile… • Netflix are best-in-class – bit.ly/1gs5n3q • System must be robust first! 01/10/2014 @danielbryantuk
  51. 51. Design for Failure • Design patterns – Timeouts / retries – Bulkheads / circuit-breakers • Inspiration – Chris Richardson (slidesha.re/1ft3vsg) – Netflix (bit.ly/1h5GMid) 01/10/2014 @danielbryantuk
  52. 52. Retries https://github.com/rholder/guava-retrying 01/10/2014 @danielbryantuk
  53. 53. Circuit-breaker 01/10/2014 @danielbryantuk https://github.com/Netflix/Hystrix https://github.com/Netflix/Hystrix/tree/master/hystrix-contrib/hystrix-javanica http://java.dzone.com/articles/hystrix-and-spring-boot http://projects.spring.io/spring-cloud/
  54. 54. Robust in the Cloud • Distributed Computing Principles – ‘For young bloods’ (bit.ly/1pKVepz) • Check your business goals – Lift and shift? – Cloud hybrid? – Cloud native? (Elastic scaling etc) 01/10/2014 @danielbryantuk
  55. 55. Antifragile Patterns: Respect the CAP Eventual consistency (ACID vs BASE) Clever caching (soft-state) http://en.wikipedia.org/wiki/CAP_theorem http://cloudshankar.blogspot.co.uk/2013/05/eventual-consistency.html 01/10/2014 @danielbryantuk
  56. 56. Antifragile Patterns: Elastic Scaling Stateless components Distributed data stores / caches Asynchronous communication Command Query Responsibility Segregation (CQRS) 01/10/2014 @danielbryantuk
  57. 57. So, Cloud Apps are ‘done’ when… Documented (just enough) Highly cohesive/loosely coupled (all the way down) Automated from commit to Cloud Resource aware Monitored thoroughly Antifragile 01/10/2014 @danielbryantuk
  58. 58. Thanks For Listening • Massive thanks to all the Open Credo team! • Questions / comments? – daniel.bryant@opencredo.com – @danielbryantuk 01/10/2014 @danielbryantuk

×