re:Invent 2022 DAT316 Build resilient applications using Amazon RDS and Aurora PostgreSQL

2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build resilient applications using Amazon RDS and Aurora PostgreSQL D A T 3 1 6 Grant McAlister Senior Principal Engineer Amazon Web Services Jim Mlodgenski Principal Database Engineer Amazon Web Services

3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Primary The case for resiliency • Everything fails at some point Availability Zone 1 Availability Zone 2 AWS Region Applications

9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Primary The case for resiliency • Everything fails at some point • Planned and unplanned Availability Zone 1 Availability Zone 2 AWS Region Applications

10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Primary The case for resiliency • Everything fails at some point • Planned and unplanned Availability Zone 1 Availability Zone 2 AWS Region Applications

11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary The case for resiliency • Everything fails at some point • Planned and unplanned Availability Zone 1 Availability Zone 2 AWS Region Applications Applications

12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary The case for resiliency • Everything fails at some point • Planned and unplanned • Does the system operate as designed Availability Zone 1 Availability Zone 2 AWS Region Applications Applications

13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. The case for resiliency • Everything fails at some point • Planned and unplanned • Does the system operate as designed Availability Zone 1 Availability Zone 2 AWS Region Applications Applications Standby Primary

16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. The case for resiliency • Everything fails at some point • Planned and unplanned • Does the system operate as designed • Grey failures Availability Zone 1 Availability Zone 2 AWS Region Applications Applications Standby Primary

17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications

18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s 1 2 3 4 5 6

19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system 1 2 3 4 5 6

20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ 1 2 3 4 5 6 writer reader reader

21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ 1 2 3 4 5 6 writer reader reader

22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ 1 2 3 4 5 6 reader

23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ 1 2 3 4 5 6 reader writer

27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ • AWS JDBC driver for faster failover 1 2 3 4 5 6 reader AWS JDBC writer

28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora with PostgreSQL compatibility Availability Zone 1 AWS Region Applications Availability Zone 2 Availability Zone 3 Aurora Storage Applications Applications • 4 of 6 write quorum across 3 AZ’s • Self repairing storage system • One or more db instances per AZ • AWS JDBC driver for faster failover 1 2 3 4 5 6 reader AWS JDBC writer

29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Amazon RDS Multi-AZ with two readable standbys • PostgreSQL replication across 3 AZ’s with a 2 of 3 quorum • Write Ahead Log written to local storage • Failover time a factor of transaction apply lag Availability Zone 3 AWS Region Applications Availability Zone 1 Primary Availability Zone 2 Standby WAL WAL WAL

30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary Amazon RDS Multi-AZ with one standby • PostgreSQL community version with easy configuration and management • High availability across two availability zones • All writes synchronously replicated to the standby • Failover a factor of checkpoints Availability Zone 1 Availability Zone 2 AWS Region Applications

31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recovery Time 3 GiB Redo Recovered in 19 seconds 10 GiB Redo Recovered in 50 seconds 30 GiB Redo Recovered in 123 seconds 0 20 40 60 80 100 120 140 160 0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 Recovery Time in Seconds (less is better) Writes / Second (more is better) RECOVERY TIME FROM CRASH UNDER LOAD Bubble size represents redo log, which must be recovered As PostgreSQL throughput goes up, so does log size and crash recovery time

32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recovery Time 3 GiB Redo Recovered in 19 seconds 10 GiB Redo Recovered in 50 seconds 30 GiB Redo Recovered in 123 seconds 0 20 40 60 80 100 120 140 160 0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 Recovery Time in Seconds (less is better) Writes / Second (more is better) RECOVERY TIME FROM CRASH UNDER LOAD Bubble size represents redo log, which must be recovered As PostgreSQL throughput goes up, so does log size and crash recovery time Amazon Aurora has no redo. Recovered in 3 seconds while maintaining greater throughput.

33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failover time 0 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200 Transactions per Second (TPS) Seconds PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds Baseline 32 seconds

34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failover time 0 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200 Transactions per Second (TPS) Seconds PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds Baseline 340 seconds

35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. RO Application Cluster Cache Management (CCM) Feature RW Application RO Application Async Invalidation & Update Availability zone 1 Availability zone 3 Availability zone 2 Aurora Storage RO RO RO RO

36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. RO Application Cluster Cache Management (CCM) Feature RW Application RO Application Async Invalidation & Update Availability zone 1 Availability zone 3 Availability zone 2 Aurora Storage RO RO RO RO apg_ccm_enabled=on

40. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failover time - Cluster Cache Management 0 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200 Transactions per Second (TPS) Seconds PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds Baseline CCM Enabled 340 seconds

41. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Failover time - Cluster Cache Management 0 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200 Transactions per Second (TPS) Seconds PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds Baseline CCM Enabled 32 seconds 340 seconds

42. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pg_prewarm • Extension available in all supported versions of PostgreSQL • Can manually load tables and indexes into cache • PostgreSQL 11 introduced auto prewarm to restore the cache after a restart or failover Standby Primary Availability Zone 1 Availability Zone 2 AWS Region

47. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pg_prewarm • Extension available in all supported versions of PostgreSQL • Can manually load tables and indexes into cache • PostgreSQL 11 introduced auto prewarm to restore the cache after a restart or failover Primary Availability Zone 1 Availability Zone 2 AWS Region

48. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Pg_prewarm • Extension available in all supported versions of PostgreSQL • Can manually load tables and indexes into cache • PostgreSQL 11 introduced auto prewarm to restore the cache after a restart or failover Primary Availability Zone 1 Availability Zone 2 AWS Region

50. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary Fault Injection • AWS Fault Injection Simulator • Service API’s/cli Availability Zone 1 Availability Zone 2 AWS Region Applications Applications

51. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary Fault Injection • AWS Fault Injection Simulator • Service API’s/cli  EC2 - stop-instances Availability Zone 1 Availability Zone 2 AWS Region Applications Applications

54. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Standby Primary Fault Injection • AWS Fault Injection Simulator • Service API’s/cli  EC2 - stop-instances  RDS - reboot-instance --force-failover  Aurora – failover-db-cluster Availability Zone 1 Availability Zone 2 AWS Region Applications Applications

55. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fault Injection • AWS Fault Injection Simulator • Service API’s/cli  EC2 - stop-instances  RDS - reboot-instance --force-failover  Aurora – failover-db-cluster Availability Zone 1 Availability Zone 2 AWS Region Applications Applications Standby Primary

58. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Timeouts • Timeouts prevent runaway queries • Timeouts can be implemented at any or all layers in the application stack Availability Zone 1 Availability Zone 2 AWS Region Applications 4 sec 10 sec 30 sec

59. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Timeouts • Timeouts prevent runaway queries • Timeouts can be implemented at any or all layers in the application stack • Improperly coordinated timeouts can cause connection storms Availability Zone 1 Availability Zone 2 AWS Region Applications 4 sec 10 sec 30 sec

60. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Timeouts • Timeouts prevent runaway queries • Timeouts can be implemented at any or all layers in the application stack • Improperly coordinated timeouts can cause connection storms • The database statement timeout needs to be the shortest duration Availability Zone 1 Availability Zone 2 AWS Region Applications 30 sec 10 sec 4 sec

re:Invent 2022 DAT316 Build resilient applications using Amazon RDS and Aurora PostgreSQL

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a re:Invent 2022 DAT316 Build resilient applications using Amazon RDS and Aurora PostgreSQL

Semelhante a re:Invent 2022 DAT316 Build resilient applications using Amazon RDS and Aurora PostgreSQL (20)

Mais de Grant McAlister

Mais de Grant McAlister (12)

Último

Último (20)

re:Invent 2022 DAT316 Build resilient applications using Amazon RDS and Aurora PostgreSQL