Microservices. The good the bad and the ugly

1. @aahoogendoorn MICROSERVICES THE GOOD, THE BAD AND THE UGLY Sander Hoogendoorn ditisagile.nl Mentoring ▪ Consulting ▪ Training ▪ Agile ▪ Software architecture ▪ Code

2. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 2 @aahoogendoorn www.ditisagile.nl Sander Hoogendoorn Me  Dad  Mentor, trainer, software architect, programmer  Books, articles, conferences Work  Owner, ditisagile.nl  CTO Klaverblad Verzekeringen  [PTO Capgemini]  [Global design authority agile Capgemini] Web  www.sanderhoogendoorn.com  www.smartusecase.com  www.speedbird9.com  @aahoogendoorn  sander@ditisagile.nl

5. @aahoogendoorn MONOLITHS Hard to deliver, even harder to test and impossible to maintain

6. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 7 @aahoogendoorn www.ditisagile.nl Monoliths Advantages  A single (layered) architecture  A single technology stack  A single code base maintained by multiple teams Disadvantages  All parts are interconnected  Many other systems are connected to your system  Hard to change, hard to maintain  Long time between releases, thereby increasing risks  Slow innovation  Hard to move to newer technologies  Doesn’t scale very well Product Account Order Customer Products Accounts Orders Customers

8. @aahoogendoorn A BRIEF HISTORY OF COMPONENTS AND SERVICES

13. @aahoogendoorn MICROSERVICES Beyond the hype?

16. @aahoogendoorn MICROSERVICES The clear benefits

18. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 19 @aahoogendoorn www.ditisagile.nl Inshort,themicroservicearchitecturalstyleisanapproachtodevelopingasingleapplicationasasuiteofsmallservices, eachrunninginitsownprocessandcommunicatingwithlightweightmechanisms,oftenanHTTPresourceAPI. Theseservicesarebuiltaroundbusinesscapabilitiesandindependentlydeployablebyfullyautomateddeploymentmachinery. Thereisabareminimumofcentralizedmanagementoftheseservices,whichmaybewrittenindifferentprogramminglanguages andusedifferentdatastoragetechnologies. MartinFowler

19. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 20 @aahoogendoorn www.ditisagile.nl Inshort,themicroservicearchitecturalstyleisanapproachtodevelopingasingleapplicationasasuiteofsmallservices, eachrunninginitsownprocessandcommunicatingwithlightweightmechanisms,oftenanHTTPresourceAPI. Theseservicesarebuiltaroundbusinesscapabilitiesandindependentlydeployablebyfullyautomateddeploymentmachinery. Thereisabareminimumofcentralizedmanagementoftheseservices,whichmaybewrittenindifferentprogramminglanguages andusedifferentdatastoragetechnologies. MartinFowler

20. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 21 @aahoogendoorn www.ditisagile.nl Monoliths. Scalability Product Account Order Customer Product Account Order Customer Product Account Order Customer

22. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 23 @aahoogendoorn www.ditisagile.nl Microservices. Scalability Product Account OrderCustomer Product CustomerCustomer Customer Account Account

23. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 24 @aahoogendoorn www.ditisagile.nl Microservices. Scalability Product Account Order Product Customer Account Account CustomerCustomer

24. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 25 @aahoogendoorn www.ditisagile.nl Monoliths. Persistence Product Account Order Customer Products Accounts Orders Customers

25. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 26 @aahoogendoorn www.ditisagile.nl Microservices. Polyglot persistence Product Account OrderCustomer MongoDB Customers MongoDB Orders ActiveDirectory Accounts Oracle Products

26. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 27 @aahoogendoorn www.ditisagile.nl Microservices. Promises  Products not projects  Scalable  Decentralized governance  Replaceable parts  High performance  Technology independent  Polyglot persistence  Easy to build  Easy to test  Easier deployment than monoliths Product Account Order Customer

27. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 28 @aahoogendoorn www.ditisagile.nl Microservices. But…  What is a microservice exactly?  How small is a microservice?  Requirements in a microservice world  Components or services  Who owns a microservice?  What technologies do you use?  What protocols do you apply?  How to define messages  How to test microservices  How to coordinate when business services run across components?  How to build deployment pipelines? Product Account Order Customer

29. @aahoogendoorn ARE MICROSERVICES A STAIRWAY TO HEAVEN?

30. @aahoogendoorn OR A HIGHWAY TO HELL?

31. @aahoogendoorn MICROSERVICES IN THE REAL WORLD

32. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 33 @aahoogendoorn www.ditisagile.nl A major Dutch insurance company We have  Most functionality on an expensive mainframe  A wide variety of large systems written in Java that are hard to maintain and to test, and that are very hard to replace  Individual systems that cover large areas of functionality, usually coupled to departments  Aging technology  No mobile strategy, allowing for new business or new services to clients, and intermediaries We need to  Get rid of the mainframe  Shorten time-to-market  Lower TCO  Uphold a fully secure systems landscape

33. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 34 @aahoogendoorn www.ditisagile.nl FORTHETHINGSWEHAVETOLEARN BEFOREWECANDOTHEM, WELEARNBYDOINGTHEM Aristotle

34. @aahoogendoorn SO WHAT DID WE LEARN?

36. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 37 @aahoogendoorn www.ditisagile.nl Questions, questions, questions Communicationarchitecture.Theglue Howdowedefineinterfacesbetweenappsandcomponent?Howdowearrangemessaging? Howdowegluetogetherrapidlychangingappsandcomponent? Applicationarchitecture EnduserfacingDifferentusers,differentfastevolvingneeds Whichtechnologyisthebestforwhichpurpose? Componentarchitecture Componentsandservicesareevolvingrapidly.Howdowedecidewhichcomponentsweneed? Howdowedealwithversioning?Howdowedealwithdistributedprocessesandtransactions?

37. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 38 @aahoogendoorn www.ditisagile.nl Our guiding principles We decided to go from here  Client thinks in business processes, so we implement business processes  We move away from the mainframe, to a new systems landscape, consisting of micro-applications and micro- components  Requirements and documentation are modeled rather than written  Applications implement a single (elementary) business process  Components serve a single purpose and offer services  Applications and components all have their own bounded context – a domain model  Applications and components will have an similar internal software architecture to facilitate ease of maintenance and allow for harvesting re-use  Communication between applications and components will use a simple open protocol - REST

38. @aahoogendoorn APPLICATION ARCHITECTURE

39. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 40 @aahoogendoorn www.ditisagile.nl Presentation Process Domain Services Outsideworld Pages Grids/Panels,Controls Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Servicegateways,Serviceclients Infoobjects/Searchobjects ComponentsRelations Dossiers Intermediaries Accounts Rates

40. @aahoogendoorn COMPONENT ARCHITECTURE

41. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 42 @aahoogendoorn www.ditisagile.nl Serviceinterface Process Domain Data/Services Outsideworld Resources Representations Usecases Flow Domainobjects,Factories/Repositories Enums/Valueobjects/Tupels/Referenceobjects Storagegateways,Storageclients Infoobjects/Searchobjects StorageRelations Dossiers Intermediaries DB2 MongoDB

42. @aahoogendoorn DESIGNING MICROSERVICES

44. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 47 @aahoogendoorn www.ditisagile.nl The single unified domain model Product Vendor Stock Order Client Delivery Payment

45. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 48 @aahoogendoorn www.ditisagile.nl Bounded contexts Product Vendor Stock Order Client Delivery Payment Product

46. @aahoogendoorn MODELING APPLICATIONS

49. @aahoogendoorn MODELING COMPONENTS

51. @aahoogendoorn AND THE REST IS COMMUNICATION (OVER HTTP)

52. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 57 @aahoogendoorn www.ditisagile.nl HTTPRETURNCODESCHEATSHEET 1**.Holdon 2**.Hereyougo 3**.Goaway 4**.Youfuckedup 5**.Ifuckedup

53. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 58 @aahoogendoorn www.ditisagile.nl HTTP GET GET  Retrieve whatever information is identified by the request URI in the form of an entity  The entity is usually a single or a list of objects (of the type provided by the service, often JSON, or XML)  GET is safe (retrieval only) and idempotent  Unfortunately there are many ways of creating GET requests (see examples below)  Returns 200 (found), possibly 400 (bad request) or 404 (not found) Examples  Get an entire collection localhost:8080/countries  Find objects in the collection localhost:8080/countries?name=“stan”  Find an object in the collection by ID localhost:8080/countries/38  Find a sub-object in the collection by ID localhost:8080/countries/38/capital  Find object in the collection by ISO localhost:8080/countries?isocode=“GRC”  Find object in the collection by ISO localhost:8080/countries/isocode/GRC  Find object in the collection by ISO localhost:8080/countries/GRC

55. @aahoogendoorn TESTING MICROSERVICES

56. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 61 @aahoogendoorn www.ditisagile.nl A service development lifecycle Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

57. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 62 @aahoogendoorn www.ditisagile.nl What to test Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design Developers Unittests Developers Q&A Testers Scenario’s&API’s Testers Scenario’s&API’s Productowner Product

58. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 63 @aahoogendoorn www.ditisagile.nl What to test automated Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design Developers Unittests Developers Q&A Testers Scenario’s&API’s Testers Scenario’s&API’s Productowner Product

59. @aahoogendoorn DEPLOYING MICROSERVICES Continuous integration, build pipelines and continuous delivery

60. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 65 @aahoogendoorn www.ditisagile.nl A typical build pipeline Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

61. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 66 @aahoogendoorn www.ditisagile.nl ProductionAcceptanceIntegrationTestDevelopment A typical build pipeline Code DeveloperTest Test IntegrationTest AcceptanceTest LivePrepare&Design

63. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 68 @aahoogendoorn www.ditisagile.nl Microservices. Building a deployment pipeline Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live Code DeveloperTest Test AcceptanceTest Acceptance Live

64. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 69 @aahoogendoorn www.ditisagile.nl Microservices. Pipeline hell? Codev.2 DeveloperTestv.2 Testv.2 AcceptanceTestv.2 Acceptancev.2 Code DeveloperTest Test AcceptanceTest Acceptance Live Testv.2 AcceptanceTestv.2 Acceptancev.2 Livev.2 DeveloperTest Test AcceptanceTest Acceptance Live Codev.3 DeveloperTestv.3 Live Codev.2

65. @aahoogendoorn DO WE REALLY NEED PROJECTS? From projects to releases to continuous delivery

70. MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL? ©2015 ditisagile.nl. All Rights Reserved 75 @aahoogendoorn www.ditisagile.nl Maintenance From projects to continuous delivery? Project MaintenanceMVP MaintenanceContinuousdelivery

71. @aahoogendoorn

72. @aahoogendoorn ALLOW YOUR TEAMS TO LEARN CONTINUOUSLY…

74. @aahoogendoorn … AND HAVE FUN

81. @aahoogendoorn THIS IS AGILE www.createspace.com/4747266 Password: agilescrum Discount code: KGNWKKWG

82. @aahoogendoorn www.sanderhoogendoorn.com www.smartusecase.com www.speedbird9.com sander@ditisagile.nl @aahoogendoorn REFERENCES AND QUESTIONS