SlideShare a Scribd company logo
1 of 32
Download to read offline
Replication Solutions for
            PostgreSQL
             Peter Eisentraut
          petere@postgresql.org
What's in a Term?
•   Replication?
•   Clustering?
•   High availability?
•   Failover?
•   Standby?

• Putting data on more than one computer

                                           2
Space of Possibilities
• Goals
  • What do you want to achieve?
• Techniques
  • How can this be implemented?
• Solutions
  • What software is available to do this?




                                             3
Goals
• High availability
• Performance
  • Read
  • Write
• Wide-area networks
• Offline peers



                              4
Goal: High Availability
• No one wants “low availability”!
• Provisions for system failures
  • Software faults
  • Hardware faults
  • External interference




                                     5
Goal: Read Performance
• Applications with:
  • many readers (e.g., library information system)
  • resource-intensive readers (e.g., data
    warehousing)
• Distribute readers on more hardware.
• Most often, one physical machine is
  enough.



                                                  6
Goal: Write Performance
• Applications with:
  • Many writers
• Distribute writers on more hardware?
  • Constraint checking, conflict resolution?!?
• Faster writing contradicts replication.
  • Partition, don't replicate!
  • RAID 0/striping is not replication – it makes
    things “worse”.
  • RAID 10 is a good idea, but not the topic here.
                                                  7
Goal: Wide-Area Networks
• Faster access across WANs
• Reading?
  • Local copies
• Writing?
  • Synchronization?




                               8
Goal: Offline Peers
• Synchronize data with laptops,
  handhelds, ...
• “Road warriors”
• May be considered very-low-latency WANs




                                        9
Techniques
• Replication
  • Master/Slave
     • Asynchronous
     • Synchronous
  • Multi-Master
     • Asynchronous
     • Synchronous
• Proxy
• Standby system
                             10
Technique: Replication
   Master/Slave Asynchronous
• High(er) availability(?)
• Read performance
  • Load spreading,
    load balancing
• Offline peers              M   asy c
                                    n    S
  (unidirectional sync.)




                                             11
Technique: Replication
    Master/Slave Synchronous
• High availibility
• Better read
  performance
• Worse write
  performance         M   sy c
                            n    S




                                     12
Technique: Replication
   Multi-Master Asynchronous
• Read performance
• Faster access across
  WANs
• Manage offline peers
                         M           M
• Requires conflict          asy c
                                n


  resolution mechanism



                                         13
Technique: Replication
       Multi-Master Synchronous
•   “Holy grail of replication”
•   High availability
•   Read performance
•   Difficult to get good
                                  M          M
    write performance                 sy c
                                        n




                                                 14
Technique: Proxy
• High availability
• Read performance
• Proxy instance should       Proxy
  be redundant
• Transparent to the
  application
                          C           C



                                          15
Technique: Standby System
• High availability




                      M   sy c
                            n    S




                                     16
Constraints
• Hardware
• Operating system
• Application




                           17
No Built-In Solution?
• FIXME




                              18
Solutions
•   Slony-I, -II
•   PGCluster
•   DBMirror
•   pgpool
•   WAL replication
•   Sequoia
•   DRBD
•   Shared storage
                             19
Solution: Slony-I
(Slony ← слоны ← elephants)
• Asynchronous master/slave replication
• Multiple slaves, cascading possible
• Particularly useful for:
  • Read performance (load balancing with pgpool)
  • Limited form of high availability
  • Offline slaves via file-based log shipping
http://www.slony.info/

                                               20
Solution: Slony-II
• Synchronous master/master
  replication?
• See Gavin Sherry's session for details




                                           21
Solution: PGCluster
• Synchronous master/master replication
• Replicates the query string
• Particularly useful for:
  • Load balancing
  • High availability



http://pgcluster.projects.postgresql.org/
                                            22
Solution: DBMirror
• Asynchronous master/slave replication
• Very simple (compared to Slony-I)
• Particularly useful for:
  • Read performance
  • Offline peers



contrib/dbmirror/ in PostgreSQL source tree
                                              23
Solution: pgpool
• Connection pool daemon for PostgreSQL
• Supports simple proxying
• Useful as frontend for Slony-I




http://pgpool.projects.postgresql.org/
                                          24
Solution: WAL Replication
• Use the “archived” WAL logs for “recovery”
  on a standby system
• Disadvantages:
  • Only full database cluster replication
  • Master and slave must be binary-compatible
  • Rather slow across network
• Useful for:
  • High availability

                                                 25
Solution: Sequoia
• Formerly C[lustered]-JDBC
• Proxy offering clustering, load
  balancing and failover services
• Particularly useful for:
  • High availability
  • Read performance
• Currently only for Java/JDBC applications
http://sequoia.continuent.org/

                                              26
Solution: DRBD
•   File system (block device) replication
•   Linux kernel module
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system


http://www.drbd.org/
                                                  27
Solution: Shared Storage
•   NAS, iSCSI, Fiberchannel, ...
•   Available from many vendors
•   Standby system
•   Useful for:
    • High availability
    • Secure any service, not just a database system
• Single storage system is a possible point of
  failure

                                                  28
Summary
• Plenty of solutions for diverse applications
• Make a (project) plan.




                                                 29
Suggestions
• Minimum for any production installation:
  • Sensible disk clustering
     • RAID 10
     • Tablespace management
     • Separate disk(s) for WAL
  • DRBD or shared storage
• Slony-I for load balancing or warehousing
• Java developers consider Sequoia


                                              30
Outlook
•   Slony-II
•   WAL replication management
•   XA support
•   More packaging efforts




                                 31
The End
Replication Solutions for PostgreSQL




                                       32

More Related Content

What's hot

Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
Denish Patel
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with Barman
Gabriele Bartolini
 

What's hot (20)

Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris Environment
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQL
 
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
Streaming Replication (Keynote @ PostgreSQL Conference 2009 Japan)
 
PostgreSQL and Linux Containers
PostgreSQL and Linux ContainersPostgreSQL and Linux Containers
PostgreSQL and Linux Containers
 
Postgres on OpenStack
Postgres on OpenStackPostgres on OpenStack
Postgres on OpenStack
 
Architecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres ClusterArchitecture for building scalable and highly available Postgres Cluster
Architecture for building scalable and highly available Postgres Cluster
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performance
 
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte DataProblems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
Problems with PostgreSQL on Multi-core Systems with MultiTerabyte Data
 
Best Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on SolarisBest Practices with PostgreSQL on Solaris
Best Practices with PostgreSQL on Solaris
 
PostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/SwitchbackPostgreSQL9.3 Switchover/Switchback
PostgreSQL9.3 Switchover/Switchback
 
Deploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQLDeploying Maximum HA Architecture With PostgreSQL
Deploying Maximum HA Architecture With PostgreSQL
 
Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016Linux internals for Database administrators at Linux Piter 2016
Linux internals for Database administrators at Linux Piter 2016
 
My experience with embedding PostgreSQL
 My experience with embedding PostgreSQL My experience with embedding PostgreSQL
My experience with embedding PostgreSQL
 
Geographically Distributed PostgreSQL
Geographically Distributed PostgreSQLGeographically Distributed PostgreSQL
Geographically Distributed PostgreSQL
 
Gluster 3.3 deep dive
Gluster 3.3 deep diveGluster 3.3 deep dive
Gluster 3.3 deep dive
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4
 
Deep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL UniverseDeep Dive into RDS PostgreSQL Universe
Deep Dive into RDS PostgreSQL Universe
 
Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2 Deploying postgre sql on amazon ec2
Deploying postgre sql on amazon ec2
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter Tuning
 
PostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with BarmanPostgreSQL Disaster Recovery with Barman
PostgreSQL Disaster Recovery with Barman
 

Viewers also liked

]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots
Klaus Hofeditz
 

Viewers also liked (9)

Introduction to pg_cheat_funcs
Introduction to pg_cheat_funcsIntroduction to pg_cheat_funcs
Introduction to pg_cheat_funcs
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
Storage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability EnvironmentsStorage Replication in High-Performance High-Availability Environments
Storage Replication in High-Performance High-Availability Environments
 
High Availability Storage (susecon2016)
High Availability Storage (susecon2016)High Availability Storage (susecon2016)
High Availability Storage (susecon2016)
 
HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08HA+DRBD+Postgres - PostgresWest '08
HA+DRBD+Postgres - PostgresWest '08
 
Development of 8.3 In India
Development of 8.3 In IndiaDevelopment of 8.3 In India
Development of 8.3 In India
 
PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!PostgreSQL9.0アップデート レプリケーションがやってきた!
PostgreSQL9.0アップデート レプリケーションがやってきた!
 
]project-open[ Screenshots
]project-open[ Screenshots ]project-open[ Screenshots
]project-open[ Screenshots
 
Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016Drbd9 and drbdmanage_june_2016
Drbd9 and drbdmanage_june_2016
 

Similar to Replication Solutions for PostgreSQL

Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
Don Demcsak
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructure
elliando dias
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
DataWorks Summit
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012
Sean Laurent
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
Qian Lin
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBD
ShapeBlue
 

Similar to Replication Solutions for PostgreSQL (20)

High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2High Scalability Toronto: Meetup #2
High Scalability Toronto: Meetup #2
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Leveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case StudyLeveraging MongoDB: An Introductory Case Study
Leveraging MongoDB: An Introductory Case Study
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph Replication
 
Petabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructurePetabyte scale on commodity infrastructure
Petabyte scale on commodity infrastructure
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpande
 
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInJay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
 
Scalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and PracticeScalable Persistent Storage for Erlang: Theory and Practice
Scalable Persistent Storage for Erlang: Theory and Practice
 
MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012MongoDB Case Study at NoSQL Now 2012
MongoDB Case Study at NoSQL Now 2012
 
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache KafkaBuilding Stream Infrastructure across Multiple Data Centers with Apache Kafka
Building Stream Infrastructure across Multiple Data Centers with Apache Kafka
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
 
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
Nagios Conference 2014 - Jeremy Rust - Avoiding Downtime Using Linux High Ava...
 
What CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBDWhat CloudStackers Need To Know About LINSTOR/DRBD
What CloudStackers Need To Know About LINSTOR/DRBD
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
 
Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)Polyglot and functional (Devoxx Nov/2011)
Polyglot and functional (Devoxx Nov/2011)
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
MongoBoston - MongoHQ
MongoBoston - MongoHQMongoBoston - MongoHQ
MongoBoston - MongoHQ
 

More from Peter Eisentraut

The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)
Peter Eisentraut
 

More from Peter Eisentraut (20)

Programming with Python and PostgreSQL
Programming with Python and PostgreSQLProgramming with Python and PostgreSQL
Programming with Python and PostgreSQL
 
Getting Started with PL/Proxy
Getting Started with PL/ProxyGetting Started with PL/Proxy
Getting Started with PL/Proxy
 
Linux distribution for the cloud
Linux distribution for the cloudLinux distribution for the cloud
Linux distribution for the cloud
 
Most Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL FeaturesMost Wanted: Future PostgreSQL Features
Most Wanted: Future PostgreSQL Features
 
Porting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQLPorting Applications From Oracle To PostgreSQL
Porting Applications From Oracle To PostgreSQL
 
Porting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQLPorting Oracle Applications to PostgreSQL
Porting Oracle Applications to PostgreSQL
 
PostgreSQL and XML
PostgreSQL and XMLPostgreSQL and XML
PostgreSQL and XML
 
XML Support: Specifications and Development
XML Support: Specifications and DevelopmentXML Support: Specifications and Development
XML Support: Specifications and Development
 
PostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie DatenbankalternativePostgreSQL: Die Freie Datenbankalternative
PostgreSQL: Die Freie Datenbankalternative
 
The Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future DevelopmentsThe Road to the XML Type: Current and Future Developments
The Road to the XML Type: Current and Future Developments
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
PostgreSQL and PL/Java
PostgreSQL and PL/JavaPostgreSQL and PL/Java
PostgreSQL and PL/Java
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
PostgreSQL News
PostgreSQL NewsPostgreSQL News
PostgreSQL News
 
Access ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-FrontendsAccess ohne Access: Freie Datenbank-Frontends
Access ohne Access: Freie Datenbank-Frontends
 
Docbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XMLDocbook: Textverarbeitung mit XML
Docbook: Textverarbeitung mit XML
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail Sy...
 
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...Collateral Damage:
Consequences of Spam and Virus Filtering for the E-Mail S...
Collateral Damage: Consequences of Spam and Virus Filtering for the E-Mail S...
 
Spaß mit PostgreSQL
Spaß mit PostgreSQLSpaß mit PostgreSQL
Spaß mit PostgreSQL
 
The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)The Common Debian Build System (CDBS)
The Common Debian Build System (CDBS)
 

Recently uploaded

Recently uploaded (20)

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 

Replication Solutions for PostgreSQL

  • 1. Replication Solutions for PostgreSQL Peter Eisentraut petere@postgresql.org
  • 2. What's in a Term? • Replication? • Clustering? • High availability? • Failover? • Standby? • Putting data on more than one computer 2
  • 3. Space of Possibilities • Goals • What do you want to achieve? • Techniques • How can this be implemented? • Solutions • What software is available to do this? 3
  • 4. Goals • High availability • Performance • Read • Write • Wide-area networks • Offline peers 4
  • 5. Goal: High Availability • No one wants “low availability”! • Provisions for system failures • Software faults • Hardware faults • External interference 5
  • 6. Goal: Read Performance • Applications with: • many readers (e.g., library information system) • resource-intensive readers (e.g., data warehousing) • Distribute readers on more hardware. • Most often, one physical machine is enough. 6
  • 7. Goal: Write Performance • Applications with: • Many writers • Distribute writers on more hardware? • Constraint checking, conflict resolution?!? • Faster writing contradicts replication. • Partition, don't replicate! • RAID 0/striping is not replication – it makes things “worse”. • RAID 10 is a good idea, but not the topic here. 7
  • 8. Goal: Wide-Area Networks • Faster access across WANs • Reading? • Local copies • Writing? • Synchronization? 8
  • 9. Goal: Offline Peers • Synchronize data with laptops, handhelds, ... • “Road warriors” • May be considered very-low-latency WANs 9
  • 10. Techniques • Replication • Master/Slave • Asynchronous • Synchronous • Multi-Master • Asynchronous • Synchronous • Proxy • Standby system 10
  • 11. Technique: Replication Master/Slave Asynchronous • High(er) availability(?) • Read performance • Load spreading, load balancing • Offline peers M asy c n S (unidirectional sync.) 11
  • 12. Technique: Replication Master/Slave Synchronous • High availibility • Better read performance • Worse write performance M sy c n S 12
  • 13. Technique: Replication Multi-Master Asynchronous • Read performance • Faster access across WANs • Manage offline peers M M • Requires conflict asy c n resolution mechanism 13
  • 14. Technique: Replication Multi-Master Synchronous • “Holy grail of replication” • High availability • Read performance • Difficult to get good M M write performance sy c n 14
  • 15. Technique: Proxy • High availability • Read performance • Proxy instance should Proxy be redundant • Transparent to the application C C 15
  • 16. Technique: Standby System • High availability M sy c n S 16
  • 17. Constraints • Hardware • Operating system • Application 17
  • 19. Solutions • Slony-I, -II • PGCluster • DBMirror • pgpool • WAL replication • Sequoia • DRBD • Shared storage 19
  • 20. Solution: Slony-I (Slony ← слоны ← elephants) • Asynchronous master/slave replication • Multiple slaves, cascading possible • Particularly useful for: • Read performance (load balancing with pgpool) • Limited form of high availability • Offline slaves via file-based log shipping http://www.slony.info/ 20
  • 21. Solution: Slony-II • Synchronous master/master replication? • See Gavin Sherry's session for details 21
  • 22. Solution: PGCluster • Synchronous master/master replication • Replicates the query string • Particularly useful for: • Load balancing • High availability http://pgcluster.projects.postgresql.org/ 22
  • 23. Solution: DBMirror • Asynchronous master/slave replication • Very simple (compared to Slony-I) • Particularly useful for: • Read performance • Offline peers contrib/dbmirror/ in PostgreSQL source tree 23
  • 24. Solution: pgpool • Connection pool daemon for PostgreSQL • Supports simple proxying • Useful as frontend for Slony-I http://pgpool.projects.postgresql.org/ 24
  • 25. Solution: WAL Replication • Use the “archived” WAL logs for “recovery” on a standby system • Disadvantages: • Only full database cluster replication • Master and slave must be binary-compatible • Rather slow across network • Useful for: • High availability 25
  • 26. Solution: Sequoia • Formerly C[lustered]-JDBC • Proxy offering clustering, load balancing and failover services • Particularly useful for: • High availability • Read performance • Currently only for Java/JDBC applications http://sequoia.continuent.org/ 26
  • 27. Solution: DRBD • File system (block device) replication • Linux kernel module • Standby system • Useful for: • High availability • Secure any service, not just a database system http://www.drbd.org/ 27
  • 28. Solution: Shared Storage • NAS, iSCSI, Fiberchannel, ... • Available from many vendors • Standby system • Useful for: • High availability • Secure any service, not just a database system • Single storage system is a possible point of failure 28
  • 29. Summary • Plenty of solutions for diverse applications • Make a (project) plan. 29
  • 30. Suggestions • Minimum for any production installation: • Sensible disk clustering • RAID 10 • Tablespace management • Separate disk(s) for WAL • DRBD or shared storage • Slony-I for load balancing or warehousing • Java developers consider Sequoia 30
  • 31. Outlook • Slony-II • WAL replication management • XA support • More packaging efforts 31
  • 32. The End Replication Solutions for PostgreSQL 32