Patterns for building resilient and scalable microservices platform on AWS
•
12 gostaram•21,888 visualizações
In this talk we explore Hailo's H2 platform under the hood taking a peek into the orchestration layer and introducing various patterns for building scalable and resilient microservices platform. We share insights about our architecture and how it evolved into a cloud agnostic self-managed system.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Patterns for building resilient and scalable microservices platform on AWS
Semelhante a Patterns for building resilient and scalable microservices platform on AWS (20)
Último
Último (20)
Patterns for building resilient and scalable microservices platform on AWS
- 1. Patterns for building resilient and scalable microservices platform on AWS Boyan Dimitrov, Platform Automation Lead @ Hailo @nathariel
- 3. Back in 2011 we started simple We quickly found out that supporting monoliths is hard: • Hard to maintain the codebase • Hard to build new features • Hard to scale the dev teams Failure to deliver business value Frontend Backend MySQL
- 4. So in 2013 we ended up doing…
- 5. At present we have • Microservices ecosystem (99.9% written in Go) • Designed specifically for the cloud – different building blocks and components will constantly be in flux, broken or unavailable • 1000+ AWS instances spanning multiple regions • 200+ services in production
- 6. The Platform under the hood
- 7. TeVPC Auto Scaling S3 OrchestrationEnv DNS Release AutoScaling Discovery Monitoring CFEC2 Route 53 Redshift ComputeEIP Routing Core Platform Provisioning Login Services Cloud Provider Whisper Config
- 8. • Lowest level building blocks • We mostly use basic PaaS components and services as they cover most of our needs • We expect every underlying component to fail and we designed for this
- 9. eu-west-1 Message Bus+ Go Services Proxy Layer C* us-east-1 Proxy Layer C* Go Services Message Bus+
- 10. eu-west-1 Proxy Layer Message Bus eu-west-1a Services eu-west-1b eu-west-1c Shared Infra RabbitMQ RabbitMQ RabbitMQ API API API Go Go Go x many C* NSQ ZK C* NSQ ZK C* NSQ ZK x many x many
- 11. • We use auto scaling groups for everything Guarantees each component can be rebuilt automatically Including our database clusters that run on ephemeral storage ( we do keep 6 copies of each piece of data in 2 regions ) • Minimum of 3 AZs in every region • Every workflow is automated • Every component has to be self healing and scalable Basic principles
- 12. • Our “cloud provider abstraction” layer • Main purpose is infrastructure and workflow automation and discovery • Has a global view of everything happening across our infrastructure • Provides additional capabilities on top of AWS • Runs in a dedicated VPCs across two regions OrchestrationEnv DNS Release AutoScalingComputeEIP Whisper
- 13. It all started by a small challenge we had to overcome: Payment providers whitelist sources
- 14. EIP Service Elastic IP Provisioning Service NAT LIVE NAT FOO 51.x.x.1 nat live 51.x.x.2 nat live 51.x.x.3 nat live 50.x.x.5 1 nat foo Maintains elastic IP pools across all our accounts and matches them against auto scaling groups and environments auto scaling group auto scaling group
- 15. We do a lot of server discovery • Both external and internal orchestration tools rely on AWS APIs for server discovery • Puppet has AWS integration for clustering infra • Exponential back-off mitigates the issue but does not solve it if you have many clients “RequestLimitExceeded”.
- 16. Compute service to the rescue • A distributed cache of all compute instances and their meta data • Powerful query API ( Very Fast!) • Main interface for creating new compute instances • Reconciles any changes in any AWS account within seconds Compute Service Other providers Internal tools External toolsServices
- 17. Everything in our platform emits events So naturally we want to capture all external events as well!
- 18. Whisper Service It’s all about event driven compute – think Lambda but within our platform Events Events Hundreds of publishers & subscribe NSQ Topics Events External sources Actions To subscribe to any new event source we have to only change a single service
- 19. What about AWS resource access?
- 20. temporary security credentials AWS Account X AWS Account Y service temporary security credentials role role • Each external orchestration service instance has a “global” view of our infrastructure • Relies heavily on STS to operate across different accounts and regions • Each service has a designated role for every account and region AWS Auth under the hood
- 21. Shared environments create contention. We decided to boost our developers productivity and give them on demand environments ENV ENV ENV
- 22. Environment Service SIE MIE Infrastructure Core Platform Single server on AWS Hundreds of servers / single AWS region CloudFormation Orchestration layer On demand environments Single Instance Environment Multi instance environment
- 23. Environment Service SIE MIE Infrastructure Core Platform Single server on AWS Hundreds of servers / single AWS region Release Service ANY ENV (PROD) Services Config *Data clone ETA: ~12 min ETA: ~40 min CloudFormation Orchestration layer On demand environments Single Instance Environment Multi instance environment
- 24. Environment Service SIE MIE Infrastructure Core Platform Single server on AWS Vagrant support Hundreds of servers / single AWS region Multi-region environments Release Service ANY ENV (PROD) Services Config *Data clone ETA: ~12 min ETA: ~40 min CloudFormation Orchestration layer On demand environments Single Instance Environment Multi instance environment
- 26. SIE Pre Prod All of this so we can do SIE MIE MIE SIE MIE SIE Live Orchestration
- 28. • The only services directly aware of our cloud provider specifics – gives us a lot of flexibility and let us introduce changes quickly • Each of them fulfills a very specific task and together create powerful workflows • Nothing else in our platform is aware of the underlying cloud layer • We did not envision being “cloud agnostic” – it just happened
- 29. Provides the most essential platform functions for every service: • Service Discovery • Service Provisioning • Routing & Load Balancing • Authentication/Authorization • Monitoring • Configuration
- 31. Provisioning Service Build Pipeline Amazon S3 Provisioning Manager Provisioning Service Docker Registry Provisioning overview Instance Instance Process Container Auto Scaling GroupAuto Scaling Group
- 32. Service deployment specifics • Each service is decoupled from the rest and deployed individually • We run multiple services on the same instance but each service is deployed in at least 3 AZs • We rely on auto scaling groups for organizing and scaling our workload • We use static partitioning to match a service to an auto scaling group and this results in non optimal resource utilisation (25% - 50%)
- 33. Deploying a service service name version auto scaling group
- 34. Coming soon: Elastic resource pools and QoS scheduling Elastic Resource Pool ECS Agent ECS Agent ECS Agent ECS Agent ECS Agent ECS Agent QoS Scheduler eu-west-1a eu-west-1b eu-west-1c AWS Cloud Provider ECS Cluster Manager instance instance instance instance instance instance
- 35. So what does this mean? Elastic resource pool 75-80% Utilization eu-west-1a eu-west-1b eu-west-1c One word – such difference! instance instance instance instance instance instance
- 36. Why building our own scheduler? • Service Priority • Service specific runtime metrics • Interference • Cloud awareness ( availability zones, pool elasticity…) Running services in a pay as you go fashion will soon be a reality as much as todays on demand compute We want a cloud-native scheduler that is aware of the cloud specifics and our microservices ecosystem:
- 37. • Self-contained units of execution • Built around business capabilities or domain objects • Small enough to be rewritten in a few days • They are all about adding business value
- 38. Service interactions – not as scary as it looks!
- 39. A microservice under the hood Logic Storage Library for abstracting service- to-service comms service-layer Handler platform-layer Self-configuring external service adapters Service • Service to service communication libs • Discovery • Configuration • A/B testing capabilities • Monitoring & Instrumentation • … and much more Any service gets for free:
- 40. Microservices are all about tooling
- 43. You need to identify your main KPIs
- 45. Thanks! Get a taxi home on us: @nathariel boyan@hailocab.com @HailoTech
Notas do Editor
- Seamless user experience
- Cloud
- We built our custom provisioning system and we started by running a number of services on a single instance Initially we were running services as normal processes on the instance but this started causing noisy neighbour problems Several months ago we gradually started moving to containers aiming for isolation and resource control capabilities.
- We want an elastic resource pool where services are scheduled on a need to basis We don’t want to manage services manually and leave that to a smart scheduler