Recomendados
Recomendados
Mais conteúdo relacionado
Semelhante a Enterprise Serverless Adoption. An Experience Report
Semelhante a Enterprise Serverless Adoption. An Experience Report (20)
Mais de SheenBrisals
Mais de SheenBrisals (10)
Último
Último (20)
Enterprise Serverless Adoption. An Experience Report
- 1. Enterprise Serverless Adoption. An Experience Report Sheen Brisals The LEGO Group
- 2. Introduction Challenges with the legacy system The reasons for change Startingwith serverless Developing with serverless Lessons and experiences The Future Agenda
- 4. Sheen Brisals Engineer. Architect. Writer. Speaker. Leader. AWS Serverless Hero @ The LEGO Group sheenbrisals sbrisals.medium.com sheen-brisals
- 6. TheLEGO Group DigitalTechnology Marketing & Channels Technology MCT Domains Squads
- 9. Painful PROD releases Pool A Pool B Drain Shutdown Deploy Test Release
- 10. Black Friday with a monolith VPC Node.js + React Elastic Beanstalk container Server contents Server contents Tax Products CRM Fulfillment Rewards VPC Gateway 503 Service Unavailable Corporate data center Server contents
- 11. Points of failures VPC Node.js + React Elastic Beanstalk container Server contents Server contents Tax Products CRM Fulfillment Rewards VPC Gateway Corporate data center Server contents Monolith One big database Fixed scaling On-premises Tightcoupling Unstable3rd party gateway Containers chaos Scaling constraints Legacy LimitedDB connections Deployment nightmare Non cloud 3rd party system No end-to-end observability
- 12. The Quest For Change
- 13. Business Expansion Reach more children - Enter new markets Development Agility Product innovation – Teamvelocity – MVP to MVP Operational Visibility Platform observability– Proactive monitoring - DevOps Business demands Minimum Viable Product Maximum Valuable Product
- 14. So, change was inevitable, but where do we start?
- 15. Too many options… Off the shelf? …and wait for every feature request? Cloud version from the same vendor? …not cloudat all but a wrapper Build own microservices? Innovative, but how do we run? docker Container Container management Isn’t that what we’re trying to avoid?
- 18. serverless Pay for compute & storage Auto scalable & highly available Cloud computing model Scale to zero & low cost No server management for you what Singlepurpose functions Managed services Nocontainerprovisioning Noloadbalancing Payas youuse Nocostforidletime Granular costvisibility Auto scaledown Individual servicescaling Service&data redundancy
- 19. serverless Ecosystem of serverless services Ideal for iterative development Optimisation at granular level Engineering diversity in teams Deeper level security & data privacy why Individual resourcelevel Multi options perservice Notjust functions Fullymanagedservices Beyondbeingaprogrammer Multipleresponsibilities Startsimpleandscaleup Event-drivenandextendable Leastprivilegeprinciple Encryption&multilayerprotection
- 21. “No one starts perfect with Serverless, but strive to be better at every iteration, and that is important.”
- 24. Transform feed Transformed feeds queue Feeds store Headless commerce platform Update data Data feeds Trigger Push Trigger Update DLQ First taste of event-driven computing Legacy commerce platform Oracle ATG RDBMS Data feeds Outperformed the legacy system!
- 25. Kinesis DataFirehose API Gateway S3 bucket Event producer Transformation and validation Backup Fan-out function Processing functions Trigger Async Errors Event ingestion & distribution Appreciated the efficiency of Kinesis! 100s of millions events ingested daily
- 26. Admin user Vouchers API Request queue Generate vouchers Vouchers store Voucher topic Email notification Email with AmazonS3 signed URL Acknowledge POST GET Async Sync Rewards generation and notification 1000s of voucher generation at a time Learned the decoupling of batch jobs and event notifications
- 27. Headless commerce platform Data feeds Trigger Push Trigger Update Push Trigger Push Trigger Product Pricing SKU DLQs 18 Event-driven data pipelines Learned the power of granularity in serverless
- 28. Customer topic New commerce platform New customer datahandler Customer created Fetch data Updatedata Reward customer handler Saved payment datahandler Fetch data Fetch data Updatedata Updatedata On-demand customer data migration Serverlessand legacyin harmony! Take bold steps, think differently & trust event-driven architecture
- 29. Shop POST GET Submit order API Submit order function Submit order queue Process order function Update status Order status API Status provider function Order cache (TTL) Order number Query status Payment providers New commerce platform User-facing checkout services evolved Serverlessfor performant user-facingAPIs Admired the serverless scaling and performance to serve customers efficiently!
- 30. The Outcome?
- 31. 10, July 2019 LEGO.com was switched to serverless
- 32. AmazonS3 CDN/WAF AWS WAF AWS Fargate AmazonElastiCache AmazonAPI Gateway AWS Lambda AWS Lambda AmazonKinesis Data Firehose AWS Lambda AmazonEvent bridge AmazonSQS AmazonDynamo DB AmazonSNS Amazon CloudWatch Third party payments ecommerce etc AWS Step Functions LEGO.com high level architecture AWS Lambda based microservices AmazonECS on Fargate
- 33. Agile, iterative, event-driven, and automated serverless development is like a gravity defying take-off. Once lifted-off, there’s no stopping!
- 34. Spring2018 Ecommerce team From… 1 team 10 engineers 1 monolith 8 application servers
- 35. July2019 Serverless team Front-end team 2 squads 20 engineers 2 repositories 1 AWS prod account 20 microservices 100 lambda functions
- 36. Early2023 payments loyalty platform content insights POS products checkout orders campaigns consumers 25+ squads 150+ engineers multiple repositories multiple AWS prod accounts many microservices many Lambda functions and growing! basket OMS store APP CSC consent accounts rewards bricks
- 37. Businessgrowth Record breaking sales weekend August 2020 November 2020 Record breaking BFCM weekend November 2019 Record breaking BFCM weekend Record sales BFCM weekend Part Serverless November 2018
- 39. The Learnings
- 40. # Accept change. Prepare to change Serverless requires a mind-shift to see the products as event-driven orchestration of managed services, knitted with infrastructure code. Sheen Brisals Mindset shift from a traditional monolith mountain thinking to a molehill microservices thinking
- 41. Serverless requires a new way of thinking, working, and running applications. Meaning, we need to change our old way of thinking, working, and running applications.
- 42. Old vs Serverless Way Old way of thinking Idea Design Build Operate Sustain Serverless way of thinking Requirements Analysis Design Development Testing Release Maintenance BigBang Incremental &Iterative Done& Dead Doing&Alive Siloed Specialists Full stack& Diverse Serverlessrequires a new way of thinking, working, and running applications. Meaning, we needto changeour old way of thinking, working, and running applications.
- 43. # Think of business domains Domain-Driven Design It’s not a book. It’s an industry!
- 44. Domain model Supple design Bounded context Anticorruption layer Distillation of core domain A system that is hard to understand is hard to change
- 45. Domain 1 Domain 3 Domain 2 Domain 4 Domain 5 Domains
- 46. Subdomains
- 49. Bounded Context BoundedContext Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Bounded Context Each teamowns &operatesits microservices Microservices
- 50. # Develop in set-pieces I just don’t think it’s that simple. Nothing is. Everything is, when you break it down! Mama Mia! Here We Go Again
- 51. A scene or sequence of scenes whose execution requires logistical planning A realistic piece of stage scenery built to stand independently as part of a stage set
- 52. A rehearsed team manoeuvre practised in training in advance of matches Parts of a model that are built separately and assembled together
- 53. Developing as Set-Pieces requires planning and execution, but can be developed and tested independently, and brought together.
- 54. Payments One monolith single service One delivery pipeline One unified complex view Subdomain view
- 55. Payments commit build test stage production One delivery pipeline
- 56. Payments Higher degree of change If a part changes frequently, how can we isolate the disruption? How can we prevent deploying everything every time? Subdomain view
- 57. Payments Set-Piece view Authorizaton Pay by Link Refunds Settlements Fraud engine Reports Notifications Subdomain view
- 58. commit build test stage production Refunds Reports Authorization commit build test stage production commit build test stage production Independent delivery pipelines
- 59. Public client Loyalty ServicePlatform SaaS CRM Data Feeder Loyalty Core Event Bus Audit Service Order Processor Auth Service Loyalty Admin Rewards Manager Customer Search SaaS Survey LEGO Data Lake NPS Provider CRM Mediator Fraud Engine Webhook
- 60. How do set-pieces work together? Event-drivenarchitecture!
- 63. # Use proven patterns Patterns help a designer get a design ‘right’ faster Patterns can offer a solution Patterns can be associated with a problem Successful designs and architecturesthat can be reused
- 65. Choreography
- 66. Choreography Event Bus Publisher Subscriber Publisher Publisher Publisher Subscriber Subscriber Subscriber No conductor Pub-sub model Service decoupling Asynchronous Idempotency Traceability Use choreography to coordinate different microservices
- 67. Orchestration
- 68. Orchestration Orchestrator Workflow Long running Coordinationof tasks Low code Pause & resume Data Process Process Process Use orchestration inside microservices Use distributed orchestration across microservices
- 69. # Build well-architected microservices Reliability Performance efficiency Operational excellence Cost optimization Security Sustainability
- 70. # Build well-architected microservices Service criticality Alarms & alerts Structuredlogging Release settings - canary Serverless Lens audit Security & privileges Data privacy measures Service documentation
- 71. Perform Solution Design Technical agility Engineering clarity Business visibility Service longevity https://bit.ly/3tPEe7D
- 72. Security & Threat Modelling Security from the start Data privacy GDPR & Compliance Service protection https://qrgo.page.link/aiQsb
- 73. # Growing engineers & teams The swiftest way to grow a company is to grow its people – Robin Sharma
- 74. organisation innovative experimentation trust motivation ownership responsible risk taking self belief self-govern fast action Provide a growth environment
- 75. certifications workshops conferences lunch& learn courses podcasts newsletters meetups hackathons open source community blogs Grow serverless knowledge pool
- 76. # Automation & DevOps Account separation Infrastructureas Code (IaC) Grow CI/CD pipeline Instil a DevOps culture Integrationtests Test in real environment Alerts and alarms Service ownership
- 79. Development Lifecycle Idea Design Build Operate Sustain You Build It, You Run It
- 80. Iterative and Incremental Approach design review agree develop Start small and evolve MVP to MVP mindset Solution first. Perfection next Iterateand improve Fast decision making
- 81. CI / CD Pipelines Repository separation GitHub Actions Squad owns their pipelines Pipeline per microservice
- 83. Alarms and Alerts Be pro-active Alert channels Near real-time Review and change as necessary
- 85. The Future
- 86. Encourage continuous learning Embrace continuous refactoring Grow serverless teams How to keep up?
- 87. Serverless Podcasts Tech blogs Conferences Serverless community Newsletters Continuous Learning patterns best practices meetups cloud news open source frameworks workshops
- 88. Continuous Improvement Continuous optimization – cost, performance, and sustainability Necessary logs, traces, and metrics Review cloud cost with the squad Architecture and DevOps practices review Documentation– APIs, runbooks, playbooks, incident reports
- 90. Code Refactoring Refactoring is a process of changing a software system. Refactoring, by Martin Fowler Pre-cloud era influenced by, Object Oriented Principles Programming Language
- 91. Serverless Refactoring Program refactor Infrastructurerefactor Refactoring for performance Refactoring for cost Adapt new features Consume better services Service rewiring Best practice refactoring
- 92. Justifying Serverless Refactoring Turn refactoring needs into business goals Keep refactoring part of development process Promote refactoring as a technology motivator
- 93. Justifying Serverless Refactoring Turn refactoring needs into business goals BusinessAPI improvements Low latency NPS Score Queue batch increase Increasedthroughput Serving more customers E n g i n e e r i n g B u s i n e s s
- 94. Justifying Serverless Refactoring Keep refactoring part of development process Set aside a sprint or two every quarter Keep a team idea board to collect ideas Discuss the benefits and get the business buy-in Monitor and circulate improvement metrics
- 95. Justifying Serverless Refactoring Promote refactoring as technology motivator New services & features New patterns New CI/CD possibilities New metrics & observability options
- 97. Takeaway Points
- 98. If you spend too much thinking about a thing, you will never get it done - BruceLee You cannot create experience, you must undergo it - Albert Camus
- 99. Understand serverless. Make a humble start Have the mind-shift. Set the guard rails Think granular. Think Set-Piece microservices Grow your serverless teams. Develop talents Continue refactoring. Accelerate business value Think sustainability. Be sustainable!