O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Cloud Cost Optimization
Szabolcs Zajdo
szabolcs.zajdo@innonic.com
Startup Studio - 5 startups
$120k MRR, 2800
clients,
40k orders/day,
99.98% uptime
$100k MRR, 1800 clients,
30k rpm, 99.99...
Me
AWS 3+ years
PHP 10+ years
Devops lead 3+ years
Cloud advisor
https://www.linkedin.com/in/szabolcszajdo/
szabolcs.zajdo...
Which top 5 services have the highest costs?
It depends on your workload and your infrastructure
Tackle the easy wins, and...
Billing & Cost
Management Dashboard
https://console.aws.amazon.com/billing/home#/
#1 - Cost Explorer
https://console.aws.amazon.com/cost-reports/home
#1 - Teevity Cloud
https://www.teevity.com
#1 - CloudCheckr
https://cloudcheckr.com/
#1 - Cloudability
https://www.cloudability.com/
#1 - Google Cloud
GCP billing export
+ BigQuery
+ Google Data Studio
Cheap but needs
manual setup
setup link
Correct instance family and size
Choose newer families
Low general load, sometime bursts → t2.medium (max 20% cpu
util)
If...
Region: EU (Ireland)
vCPU ECU Memory Linux
price
Price /
ECU
Geekben
ch
Cost of 1k
GeekBench
m3.large 2 6.5 7.5 $0.146 $0....
Region: EU (Ireland)
vCPU ECU Memory Linux
price
Price /
ECU
Geekben
ch
Cost of 1k
GeekBench
m3.large 2 6.5 7.5 $0.146 $0....
● Test/dev EC2, RDS, etc. instances
● Unattached EBS volumes
● Old EBS snapshots
● Too big EBS volumes
● S3 trash (but S3 ...
● CloudWatch: notifications or automated shutdowns
● Janitor Monkey by Netflix
● EC2 unused resources
script: link
#3 - Fr...
AWS console or AWS CLI
https://www.cloudconformity.com/conformity-rules/EBS/unused-ebs-
volumes.html
#3 - Remove Unattache...
#3 - CloudCheckr idle resources scanner
If you have traffic spikes or differences in
daily/weekly/monthly traffic
Preparation: robustness, stateless, fault tolera...
Google Cloud Preemptible VMs
MS Azure Low-priority VMs (from 2017.05)
One of the biggest cost saving opportunity (EC2 -50-...
bigger instances more expensive, higher price
volatility
#5 - Spot experiences
bigger instances more expensive, higher price
volatility
#5 - Spot experiences
difference in instance families
#5 - Spot experiences
Newer instance families have higher volatility
#5 - Spot experiences
Use more AZs
Use smaller instances
Use multiple instance types
Bid higher (on-demand price, or even higher)
#5 Spot termin...
For test/dev/stage environments: manually
#5 - Create spot manually
https://github.com/cristim/autospotting
Automatically replaces AWS EC2 instances in
AutoScaling groups with identically co...
https://github.com/mapbox/spotswap
Manages spot priceouts, by activating backup on-
demand capacity
1 spot ASG, 1 on-deman...
1. Instance receives 2-minute termination notification
2. Tags itself
3. Lambda function runs every minute
4. If finds spo...
1 spot, 1 on-demand ASG, same ELB
On-demand always on, with 2 instances
Spot ASG: 60% CPU util
On-demand ASG: 70% CPU util...
#5 - Spot fleet + autoscaling
#5 - Spot fleet + autoscaling
#5 - Spotinst
Multi-cloud (AWS, GCP, Azure)
Fallback to on-demand
Handles multiple instance types
Predictive algorithms to...
Vertical vs horizontal
#6 RDS autoscaling
Poor man’s autoscaling
db.r3.large -> db.r3.xlarge -> db.r3.large
With multi-AZ only few minutes downtime
Single-AZ 10+ mi...
Prerequisite: read-write separation - 1 write server, N
read servers
● Application modification (Shoprenter ~100 work
hour...
For RDS MySQL/Mariadb/MsSql/Postgres:
https://github.com/bitbull-team/aws-rds-autoscaling
For RDS Aurora:
Built-in AWS RDS...
For RDS MySQL/Mariadb/MsSql/Postgres:
https://github.com/bitbull-team/aws-rds-autoscaling
Lambda + Route53
For RDS Aurora:...
#6 - Aurora replicas Auto Scaling Policy
#6 - Aurora replicas Auto Scaling Policy
https://www.myelastic.io/
- zero downtime
- warm-up
- in production for 6+ months
- nighttime 2, daytime 5 replicas
- Rout...
#6 - MyElastic RDS autoscaling
GCP Commited use discounts
Azure Reserved VM instances
1 year – 25-40%
3 year – 30-66% (RDS Aurora 45-66%)
- vendor lock-i...
determine instances and route causes
CloudWatch or Cloudability
Check ELB log in S3
Best practices:
●Static contents → CDN...
Replace CloudFront / Google CDN with cheaper CDN
provider
CDN77 – 75% cheaper, no cost for requests
https://www.cdn77.com/...
Extra charges for transfer between Availability Zones.
Inside AZ no transfer costs.
If more than 4 9-s (99.99%+) unnecessa...
For dev/test/stage instances
If they run less than 8 hours a day.
More than 8 hours -> Spot instances
Free: Cloudcycler, F...
Solution offered by AWS
#11 - EC2 Scheduler
If you use RDS MySQL/PostgreSQL/MariaDB
RDS -> multiAZ failover instance unreachable
Aurora -> multiAZ failover instance i...
#1 Monitor your costs (Cost Explorer, CloudCheckr)
#2 Choose correct instances
#3 Delete unneaded resources
#4 EC2 autosca...
Useful link: https://www.slideshare.net/BobAndMary4Ever/aws-cost-control
RDS auto-scaling: https://www.myelastic.io/
Thank...
Cloud cost optimization (AWS, GCP)
Cloud cost optimization (AWS, GCP)
Cloud cost optimization (AWS, GCP)
Terminou este documento.
Transfira e leia offline.
Próximos SlideShares
What to Upload to SlideShare
Avançar
Próximos SlideShares
What to Upload to SlideShare
Avançar
Transfira para ler offline e ver em ecrã inteiro.

Compartilhar

Cloud cost optimization (AWS, GCP)

Baixar para ler offline

How to optimize your cloud costs on Amazon Web Services (AWS), Google Cloud Platform and Microsoft Azure.

Livros relacionados

Gratuito durante 30 dias do Scribd

Ver tudo
  • Seja a primeira pessoa a gostar disto

Cloud cost optimization (AWS, GCP)

  1. 1. Cloud Cost Optimization Szabolcs Zajdo szabolcs.zajdo@innonic.com
  2. 2. Startup Studio - 5 startups $120k MRR, 2800 clients, 40k orders/day, 99.98% uptime $100k MRR, 1800 clients, 30k rpm, 99.99%+ uptime 3 AWS, 3 Google Cloud prod workload ~$17k AWS, $2k Google Cloud spend Innonic
  3. 3. Me AWS 3+ years PHP 10+ years Devops lead 3+ years Cloud advisor https://www.linkedin.com/in/szabolcszajdo/ szabolcs.zajdo@innonic.com
  4. 4. Which top 5 services have the highest costs? It depends on your workload and your infrastructure Tackle the easy wins, and those with the highest benefit #1 Monitor your costs
  5. 5. Billing & Cost Management Dashboard https://console.aws.amazon.com/billing/home#/
  6. 6. #1 - Cost Explorer https://console.aws.amazon.com/cost-reports/home
  7. 7. #1 - Teevity Cloud https://www.teevity.com
  8. 8. #1 - CloudCheckr https://cloudcheckr.com/
  9. 9. #1 - Cloudability https://www.cloudability.com/
  10. 10. #1 - Google Cloud GCP billing export + BigQuery + Google Data Studio Cheap but needs manual setup setup link
  11. 11. Correct instance family and size Choose newer families Low general load, sometime bursts → t2.medium (max 20% cpu util) If you need memory →r3, r4, m3, m4 – for relational and NoSQL databases, depending on if the database size fits in memory! (MySQL, RDS, Redis) Every other general case → c4, c5 (new!) #2 Choose correct instances
  12. 12. Region: EU (Ireland) vCPU ECU Memory Linux price Price / ECU Geekben ch Cost of 1k GeekBench m3.large 2 6.5 7.5 $0.146 $0.022 2441 $0.060 c3.large 2 7 3.75 $0.120 $0.017 2875 $0.042 m4.large 2 6.5 8 $0.111 $0.017 2881 $0.039 c4.large 2 8 3.75 $0.113 $0.014 3073 $0.037 m5.large 2 7.5 8 $0.107 $0.014 3549 $0.030 c5.large 2 8 4 $0.096 $0.012 3334 $0.029
  13. 13. Region: EU (Ireland) vCPU ECU Memory Linux price Price / ECU Geekben ch Cost of 1k GeekBench m3.large 2 6.5 7.5 $0.146 $0.022 2441 $0.060 c3.large 2 7 3.75 $0.120 $0.017 2875 $0.042 m4.large 2 6.5 8 $0.111 $0.017 2881 $0.039 c4.large 2 8 3.75 $0.113 $0.014 3073 $0.037 m5.large 2 7.5 8 $0.107 $0.014 3549 $0.030 c5.large 2 8 4 $0.096 $0.012 3334 $0.029 C5.large: - 25% cheaper than c4.large / compute unit - 50% cheaper than m3.large / compute unit
  14. 14. ● Test/dev EC2, RDS, etc. instances ● Unattached EBS volumes ● Old EBS snapshots ● Too big EBS volumes ● S3 trash (but S3 is cheap) #3 Search and delete unneaded resources
  15. 15. ● CloudWatch: notifications or automated shutdowns ● Janitor Monkey by Netflix ● EC2 unused resources script: link #3 - Free solutions
  16. 16. AWS console or AWS CLI https://www.cloudconformity.com/conformity-rules/EBS/unused-ebs- volumes.html #3 - Remove Unattached EBS volumes
  17. 17. #3 - CloudCheckr idle resources scanner
  18. 18. If you have traffic spikes or differences in daily/weekly/monthly traffic Preparation: robustness, stateless, fault tolerant. #4 EC2 autoscaling
  19. 19. Google Cloud Preemptible VMs MS Azure Low-priority VMs (from 2017.05) One of the biggest cost saving opportunity (EC2 -50-80%, GCP -80%, Azure -80%) Spot market: instance type + availability zone Fixed priced vs bidding Termination: continuously vs 24-hour limit #5 EC2 Spot instances
  20. 20. bigger instances more expensive, higher price volatility #5 - Spot experiences
  21. 21. bigger instances more expensive, higher price volatility #5 - Spot experiences
  22. 22. difference in instance families #5 - Spot experiences
  23. 23. Newer instance families have higher volatility #5 - Spot experiences
  24. 24. Use more AZs Use smaller instances Use multiple instance types Bid higher (on-demand price, or even higher) #5 Spot termination mitigation
  25. 25. For test/dev/stage environments: manually #5 - Create spot manually
  26. 26. https://github.com/cristim/autospotting Automatically replaces AWS EC2 instances in AutoScaling groups with identically configured For dev/test/stage environments #5 - Autospotting
  27. 27. https://github.com/mapbox/spotswap Manages spot priceouts, by activating backup on- demand capacity 1 spot ASG, 1 on-demand ASG, 1 Lambda function For prod environments too #5 - Spotswap
  28. 28. 1. Instance receives 2-minute termination notification 2. Tags itself 3. Lambda function runs every minute 4. If finds spot tags, scales up on-demand ASG #5 - Spotswap
  29. 29. 1 spot, 1 on-demand ASG, same ELB On-demand always on, with 2 instances Spot ASG: 60% CPU util On-demand ASG: 70% CPU util Max downtime: ~10 mins #5 - Poor man’s script
  30. 30. #5 - Spot fleet + autoscaling
  31. 31. #5 - Spot fleet + autoscaling
  32. 32. #5 - Spotinst Multi-cloud (AWS, GCP, Azure) Fallback to on-demand Handles multiple instance types Predictive algorithms to identify spot termination Promises no downtime But you get a black-box, eg. Spotinst has it’s own ASG (Elastigroup) They charge 20% of savings
  33. 33. Vertical vs horizontal #6 RDS autoscaling
  34. 34. Poor man’s autoscaling db.r3.large -> db.r3.xlarge -> db.r3.large With multi-AZ only few minutes downtime Single-AZ 10+ minutes downtime https://cloudiamo.com/2016/12/18/scaling-a-rds-instance-vertically-automatically/ #6 - Vertical RDS autoscaling
  35. 35. Prerequisite: read-write separation - 1 write server, N read servers ● Application modification (Shoprenter ~100 work hour), or ●ProxySQL #6 - Horizontal RDS autoscaling
  36. 36. For RDS MySQL/Mariadb/MsSql/Postgres: https://github.com/bitbull-team/aws-rds-autoscaling For RDS Aurora: Built-in AWS RDS auto scaling (new!), but still not zero downtime when scaling down #6 - Horizontal RDS autoscaling
  37. 37. For RDS MySQL/Mariadb/MsSql/Postgres: https://github.com/bitbull-team/aws-rds-autoscaling Lambda + Route53 For RDS Aurora: Built-in AWS RDS auto scaling (new!), but still not zero downtime when scaling down #6 - Horizontal RDS autoscaling
  38. 38. #6 - Aurora replicas Auto Scaling Policy
  39. 39. #6 - Aurora replicas Auto Scaling Policy
  40. 40. https://www.myelastic.io/ - zero downtime - warm-up - in production for 6+ months - nighttime 2, daytime 5 replicas - Route53 traffic policies weighted rules #6 - MyElastic RDS autoscaling
  41. 41. #6 - MyElastic RDS autoscaling
  42. 42. GCP Commited use discounts Azure Reserved VM instances 1 year – 25-40% 3 year – 30-66% (RDS Aurora 45-66%) - vendor lock-in - Modification permitted only within instance family For EC2/GCE → spot instances/preemtible Vms if you can do it, 2x bigger discount and no loyalty For other services (RDS, Elasticache), or databases running on EC2 (no robustness) #7 AWS Reserved instances
  43. 43. determine instances and route causes CloudWatch or Cloudability Check ELB log in S3 Best practices: ●Static contents → CDN ●Gzip ●Remove Fix IP ●Client side caching #8 Data transfer costs
  44. 44. Replace CloudFront / Google CDN with cheaper CDN provider CDN77 – 75% cheaper, no cost for requests https://www.cdn77.com/ http://cdncomparison.com/ Wildcard domain → L7Cache #9 Cheaper CDN
  45. 45. Extra charges for transfer between Availability Zones. Inside AZ no transfer costs. If more than 4 9-s (99.99%+) unnecessary → use instances only in 1 AZ: every services (EC2, RDS, Elasticache, etc.) #10 AZ transfer costs
  46. 46. For dev/test/stage instances If they run less than 8 hours a day. More than 8 hours -> Spot instances Free: Cloudcycler, Flywheel Freemium: ParkMyCloud #11 Schedule EC2 instances
  47. 47. Solution offered by AWS #11 - EC2 Scheduler
  48. 48. If you use RDS MySQL/PostgreSQL/MariaDB RDS -> multiAZ failover instance unreachable Aurora -> multiAZ failover instance is readable Aurora has higher reserved discounts #12 Replace RDS with Aurora
  49. 49. #1 Monitor your costs (Cost Explorer, CloudCheckr) #2 Choose correct instances #3 Delete unneaded resources #4 EC2 autoscaling #5 Spot instances #6 RDS autoscaling (horizontal) #7 Reserved instances (managed services) #8 Data transfer costs #9 Reduce CDN costs #10 AZ transfer costs #11 Schedule EC2 instances #12 Replace RDS with Aurora Summary
  50. 50. Useful link: https://www.slideshare.net/BobAndMary4Ever/aws-cost-control RDS auto-scaling: https://www.myelastic.io/ Thank you!

How to optimize your cloud costs on Amazon Web Services (AWS), Google Cloud Platform and Microsoft Azure.

Vistos

Vistos totais

772

No Slideshare

0

De incorporações

0

Número de incorporações

9

Ações

Baixados

34

Compartilhados

0

Comentários

0

Curtir

0

×