SlideShare a Scribd company logo
1 of 43
Download to read offline
Scaling with Postgres

                                              Robert Treat
                            Percona Performance Conference
                                                     2009




Wednesday, April 22, 2009
Disclaimer

      Every “really big” site uses custom solutions.
    Don’t expect to get there using out of the box
   software. You don’t have to reinvent the wheel.
                    But you may have to build the car.




Wednesday, April 22, 2009
Who am I ?




Wednesday, April 22, 2009
Who am I ?

                     OmniTI
                     TB+ databases
                     top 100 websites
                     Oracle/MySQL/Postgres
                            Not necessarily in that order :-)



Wednesday, April 22, 2009
Scaling Databases 101




Wednesday, April 22, 2009
Scaling Databases 101
                     Implementation




Wednesday, April 22, 2009
Scaling Databases 101
                     Implementation
                            Tuning
                            Optimize Queries
                            Optimize Schema




Wednesday, April 22, 2009
Scaling Databases 101
                     Implementation
                            Tuning
                            Optimize Queries
                            Optimize Schema
                              Throw Hardware At It
                              Caching



Wednesday, April 22, 2009
Scaling Databases 101
                     Implementation
                            Tuning
                            Optimize Queries
                            Optimize Schema
                              Throw Hardware At It
                              Caching
                                 Replication

Wednesday, April 22, 2009
Scaling Databases 101
                     Implementation
                            Tuning
                            Optimize Queries
                            Optimize Schema
                              Throw Hardware At It
                              Caching
                                 Replication
                                     Federation
Wednesday, April 22, 2009
Implementation

                            Normalization
                            Foreign Keys
                            Relational Modeling




Wednesday, April 22, 2009
Implementation

                            Normalization
                            Foreign Keys
                            Relational Modeling




                            Mostly Irrelevant
Wednesday, April 22, 2009
Implementation




Wednesday, April 22, 2009
Implementation
                            Monitoring
                              nagios, munin, etc..
                                 check_postgres




Wednesday, April 22, 2009
Implementation
                            Monitoring
                              nagios, munin, etc..
                                 check_postgres

                            Trending
                              cacti, noit
                                 watch your hardware!


Wednesday, April 22, 2009
Tuning




Wednesday, April 22, 2009
Tuning
                            Get the Basics Right
                              http://wiki.postgresql.org/wiki/
                              Tuning_Your_PostgreSQL_Server




Wednesday, April 22, 2009
Tuning
                            Get the Basics Right
                              http://wiki.postgresql.org/wiki/
                              Tuning_Your_PostgreSQL_Server
                            Someday
                              Xlogs, Checkpoints
                              Vacuum Strategy
                              Costs and other Minutia

Wednesday, April 22, 2009
Optimize Queries

                            explain
                            explain analyze
                            explain analyze verbose




Wednesday, April 22, 2009
Optimize Queries


                            writing queries tuned to your
                            databases planner




Wednesday, April 22, 2009
Optimize Schema




Wednesday, April 22, 2009
Optimize Schema
                            indexing
                              partial, functional
                            fix data types
                            add/remove tables
                            de-normalize (maybe)




Wednesday, April 22, 2009
Why is this important?

                            Postgres is good at scaling vertically
                            Flawed database schema hides the
                            real problems




Wednesday, April 22, 2009
Why is this unimportant?

                            Optimizing is often a trade off of time
                            vs. money
                            It is not an ultimate solution




Wednesday, April 22, 2009
Throw Hardware At It
                            Find your bottlenecks
                              Memory / RAM
                              CPU
                              I/O, DISKS
                                 usually the biggest problem




Wednesday, April 22, 2009
Can you take me higher?

                            32 Cores
                            128 GB+ RAM
                            Spindles out the wazoo
                              tablespaces




Wednesday, April 22, 2009
Caching




Wednesday, April 22, 2009
Caching
                            Internal
                               materialized views




Wednesday, April 22, 2009
Caching
                            Internal
                               materialized views
                            External
                               memcached, et al.
                               static files




Wednesday, April 22, 2009
Oh, the places you’ll go

                            Nothing Fancy or Exciting
                            Solid Engineering
                            Understanding needs / limits




Wednesday, April 22, 2009
Oh, the places you’ll go

                            Nothing Fancy or Exciting
                            Solid Engineering
                            Understanding needs / limits




                            Top 1000+ Website
Wednesday, April 22, 2009
Replication
                            Ignore the hype
                              Triggers won’t destroy your site
                              Scripting solves a lot of config
                              issues
                              Telling your DBA to do his !@#$ job
                              solves config issues too
                              Add-on is FUD for Modular

Wednesday, April 22, 2009
Replication

                            Truth in advertising
                               Oracle does it better
                               MySQL does it simpler




Wednesday, April 22, 2009
Replication




Wednesday, April 22, 2009
Replication
                     Scaling Reads
                            Master, many slaves
                            Slony, Londiste, Bucardo, PGPoolII




Wednesday, April 22, 2009
Replication
                     Scaling Reads
                            Master, many slaves
                            Slony, Londiste, Bucardo, PGPoolII
                     Scaling Writes
                            Mostly impossible
                            Bucardo, PGPoolII


Wednesday, April 22, 2009
What is Federation?

                       A process for eliminating performance
                      and/or scaleability as a requirement for
                                  your database




Wednesday, April 22, 2009
Federation




Wednesday, April 22, 2009
Federation

                     MySQL software has no federation
                     advantage




Wednesday, April 22, 2009
Federation

                     MySQL software has no federation
                     advantage
                     MySQL community has large federation
                     advantage




Wednesday, April 22, 2009
Federation
                     Things to consider
                            Horizontal Partitioning
                            Application Based Partitioning
                            BASE
                               golconde, gearman




Wednesday, April 22, 2009
El Fin

                              www.xzilla.net
                             www.omniti.com
                            www.postgresql.us




Wednesday, April 22, 2009
Postgres BOF


                     Tonight @ 7:30 PM
                     Location @ Camino Real




Wednesday, April 22, 2009

More Related Content

Viewers also liked

How to ChoosHow to Choose A Web Application Framework
How to ChoosHow to Choose A Web Application FrameworkHow to ChoosHow to Choose A Web Application Framework
How to ChoosHow to Choose A Web Application Frameworkelliando dias
 
Some Important slide to be remember for your success
Some Important slide to be remember for your successSome Important slide to be remember for your success
Some Important slide to be remember for your successAnil Gahlot
 
シニアの予備校|ハローワーク未満
シニアの予備校|ハローワーク未満シニアの予備校|ハローワーク未満
シニアの予備校|ハローワーク未満ebina yohichi
 
PHP and Rich Internet Applications
PHP and Rich Internet ApplicationsPHP and Rich Internet Applications
PHP and Rich Internet Applicationselliando dias
 
Establishing your district's relationship with google
Establishing your district's relationship with googleEstablishing your district's relationship with google
Establishing your district's relationship with googleTim Golden
 
Test-driven Development no Rails
Test-driven Development no RailsTest-driven Development no Rails
Test-driven Development no Railselliando dias
 
Virtual Reality: An Introduction
Virtual Reality:  An IntroductionVirtual Reality:  An Introduction
Virtual Reality: An Introductionelliando dias
 
Step by Step Guide for building a simple Struts Application
Step by Step Guide for building a simple Struts ApplicationStep by Step Guide for building a simple Struts Application
Step by Step Guide for building a simple Struts Applicationelliando dias
 
Ruby Day Kraków: Full Text Search with Ferret
Ruby Day Kraków: Full Text Search with FerretRuby Day Kraków: Full Text Search with Ferret
Ruby Day Kraków: Full Text Search with Ferretelliando dias
 
Css3 paul trani
Css3 paul traniCss3 paul trani
Css3 paul traniCarl Ford
 
初めてのハローワークテスト1
初めてのハローワークテスト1初めてのハローワークテスト1
初めてのハローワークテスト1ebina yohichi
 
Distributed Data processing in a Cloud
Distributed Data processing in a CloudDistributed Data processing in a Cloud
Distributed Data processing in a Cloudelliando dias
 
Building A Mini Google High Performance Computing In Ruby Presentation 1
Building A Mini Google  High Performance Computing In Ruby Presentation 1Building A Mini Google  High Performance Computing In Ruby Presentation 1
Building A Mini Google High Performance Computing In Ruby Presentation 1elliando dias
 
Bridge wave 4gwe-miami_02feb(3)
Bridge wave   4gwe-miami_02feb(3)Bridge wave   4gwe-miami_02feb(3)
Bridge wave 4gwe-miami_02feb(3)Carl Ford
 
FUSE (Filesystem in Userspace) on OpenSolaris
FUSE (Filesystem in Userspace) on OpenSolarisFUSE (Filesystem in Userspace) on OpenSolaris
FUSE (Filesystem in Userspace) on OpenSolariselliando dias
 
AR Class Preferenc
AR Class PreferencAR Class Preferenc
AR Class PreferencD Houseman
 
KISD RSS Intro
KISD RSS IntroKISD RSS Intro
KISD RSS IntroD Houseman
 
Infrastructure Engineering
Infrastructure EngineeringInfrastructure Engineering
Infrastructure Engineeringelliando dias
 
Ruby on Windows - HOW TO install and set up
Ruby on Windows - HOW TO install and set upRuby on Windows - HOW TO install and set up
Ruby on Windows - HOW TO install and set upTim Golden
 

Viewers also liked (20)

How to ChoosHow to Choose A Web Application Framework
How to ChoosHow to Choose A Web Application FrameworkHow to ChoosHow to Choose A Web Application Framework
How to ChoosHow to Choose A Web Application Framework
 
Some Important slide to be remember for your success
Some Important slide to be remember for your successSome Important slide to be remember for your success
Some Important slide to be remember for your success
 
シニアの予備校|ハローワーク未満
シニアの予備校|ハローワーク未満シニアの予備校|ハローワーク未満
シニアの予備校|ハローワーク未満
 
PHP and Rich Internet Applications
PHP and Rich Internet ApplicationsPHP and Rich Internet Applications
PHP and Rich Internet Applications
 
Establishing your district's relationship with google
Establishing your district's relationship with googleEstablishing your district's relationship with google
Establishing your district's relationship with google
 
Test-driven Development no Rails
Test-driven Development no RailsTest-driven Development no Rails
Test-driven Development no Rails
 
Virtual Reality: An Introduction
Virtual Reality:  An IntroductionVirtual Reality:  An Introduction
Virtual Reality: An Introduction
 
Step by Step Guide for building a simple Struts Application
Step by Step Guide for building a simple Struts ApplicationStep by Step Guide for building a simple Struts Application
Step by Step Guide for building a simple Struts Application
 
Ruby Day Kraków: Full Text Search with Ferret
Ruby Day Kraków: Full Text Search with FerretRuby Day Kraków: Full Text Search with Ferret
Ruby Day Kraków: Full Text Search with Ferret
 
Css3 paul trani
Css3 paul traniCss3 paul trani
Css3 paul trani
 
Hfc01 presentacion
Hfc01 presentacionHfc01 presentacion
Hfc01 presentacion
 
初めてのハローワークテスト1
初めてのハローワークテスト1初めてのハローワークテスト1
初めてのハローワークテスト1
 
Distributed Data processing in a Cloud
Distributed Data processing in a CloudDistributed Data processing in a Cloud
Distributed Data processing in a Cloud
 
Building A Mini Google High Performance Computing In Ruby Presentation 1
Building A Mini Google  High Performance Computing In Ruby Presentation 1Building A Mini Google  High Performance Computing In Ruby Presentation 1
Building A Mini Google High Performance Computing In Ruby Presentation 1
 
Bridge wave 4gwe-miami_02feb(3)
Bridge wave   4gwe-miami_02feb(3)Bridge wave   4gwe-miami_02feb(3)
Bridge wave 4gwe-miami_02feb(3)
 
FUSE (Filesystem in Userspace) on OpenSolaris
FUSE (Filesystem in Userspace) on OpenSolarisFUSE (Filesystem in Userspace) on OpenSolaris
FUSE (Filesystem in Userspace) on OpenSolaris
 
AR Class Preferenc
AR Class PreferencAR Class Preferenc
AR Class Preferenc
 
KISD RSS Intro
KISD RSS IntroKISD RSS Intro
KISD RSS Intro
 
Infrastructure Engineering
Infrastructure EngineeringInfrastructure Engineering
Infrastructure Engineering
 
Ruby on Windows - HOW TO install and set up
Ruby on Windows - HOW TO install and set upRuby on Windows - HOW TO install and set up
Ruby on Windows - HOW TO install and set up
 

Similar to Scaling with Postgres

Plugin Testing
Plugin TestingPlugin Testing
Plugin TestingTim Moore
 
Object Oriented Css For High Performance Websites And Applications
Object Oriented Css For High Performance Websites And ApplicationsObject Oriented Css For High Performance Websites And Applications
Object Oriented Css For High Performance Websites And ApplicationsPerconaPerformance
 
Ruby On Rails Presentation Barcamp Antwerp.Key
Ruby On Rails Presentation Barcamp Antwerp.KeyRuby On Rails Presentation Barcamp Antwerp.Key
Ruby On Rails Presentation Barcamp Antwerp.KeyBert Goethals
 
Apple Store, London 2009 02 25
Apple Store, London 2009 02 25Apple Store, London 2009 02 25
Apple Store, London 2009 02 25ProjectWizards
 
intl me this, intl me that
intl me this, intl me thatintl me this, intl me that
intl me this, intl me thatAndrei Zmievski
 
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action CachingAccelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action Cachingelliando dias
 
iPhone Development Overview
iPhone Development OverviewiPhone Development Overview
iPhone Development OverviewTom Adams
 
IT Depends: Custom vs Packaged Software
IT Depends: Custom vs Packaged SoftwareIT Depends: Custom vs Packaged Software
IT Depends: Custom vs Packaged Software★ Selcuk Atli
 
Smalltalk Metaprogramming supports Probabilistic Program Analysis
Smalltalk Metaprogramming supports Probabilistic Program AnalysisSmalltalk Metaprogramming supports Probabilistic Program Analysis
Smalltalk Metaprogramming supports Probabilistic Program AnalysisESUG
 
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em português
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em portuguêsAgile Development no Yahoo! com Scrum - Campus Party 2009 - em português
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em portuguêsAntonio Silveira
 
Agile Is the New Waterfall
Agile Is the New WaterfallAgile Is the New Waterfall
Agile Is the New WaterfallNaresh Jain
 
Google Developer Days Brazil 2009 - Keynote
Google Developer Days Brazil 2009 -  KeynoteGoogle Developer Days Brazil 2009 -  Keynote
Google Developer Days Brazil 2009 - KeynotePatrick Chanezon
 
Managing And Optimizing Memory Usage
Managing And Optimizing Memory UsageManaging And Optimizing Memory Usage
Managing And Optimizing Memory UsageJohn Wilker
 

Similar to Scaling with Postgres (18)

Plugin Testing
Plugin TestingPlugin Testing
Plugin Testing
 
Testing In Agile
Testing In AgileTesting In Agile
Testing In Agile
 
Becoming Indie
Becoming IndieBecoming Indie
Becoming Indie
 
Becoming Indie
Becoming IndieBecoming Indie
Becoming Indie
 
Object Oriented Css For High Performance Websites And Applications
Object Oriented Css For High Performance Websites And ApplicationsObject Oriented Css For High Performance Websites And Applications
Object Oriented Css For High Performance Websites And Applications
 
Cloudera Desktop
Cloudera DesktopCloudera Desktop
Cloudera Desktop
 
Ruby On Rails Presentation Barcamp Antwerp.Key
Ruby On Rails Presentation Barcamp Antwerp.KeyRuby On Rails Presentation Barcamp Antwerp.Key
Ruby On Rails Presentation Barcamp Antwerp.Key
 
Apple Store, London 2009 02 25
Apple Store, London 2009 02 25Apple Store, London 2009 02 25
Apple Store, London 2009 02 25
 
intl me this, intl me that
intl me this, intl me thatintl me this, intl me that
intl me this, intl me that
 
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action CachingAccelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
 
iPhone Development Overview
iPhone Development OverviewiPhone Development Overview
iPhone Development Overview
 
IT Depends: Custom vs Packaged Software
IT Depends: Custom vs Packaged SoftwareIT Depends: Custom vs Packaged Software
IT Depends: Custom vs Packaged Software
 
Alternative Databases
Alternative DatabasesAlternative Databases
Alternative Databases
 
Smalltalk Metaprogramming supports Probabilistic Program Analysis
Smalltalk Metaprogramming supports Probabilistic Program AnalysisSmalltalk Metaprogramming supports Probabilistic Program Analysis
Smalltalk Metaprogramming supports Probabilistic Program Analysis
 
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em português
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em portuguêsAgile Development no Yahoo! com Scrum - Campus Party 2009 - em português
Agile Development no Yahoo! com Scrum - Campus Party 2009 - em português
 
Agile Is the New Waterfall
Agile Is the New WaterfallAgile Is the New Waterfall
Agile Is the New Waterfall
 
Google Developer Days Brazil 2009 - Keynote
Google Developer Days Brazil 2009 -  KeynoteGoogle Developer Days Brazil 2009 -  Keynote
Google Developer Days Brazil 2009 - Keynote
 
Managing And Optimizing Memory Usage
Managing And Optimizing Memory UsageManaging And Optimizing Memory Usage
Managing And Optimizing Memory Usage
 

More from elliando dias

Clojurescript slides
Clojurescript slidesClojurescript slides
Clojurescript slideselliando dias
 
Why you should be excited about ClojureScript
Why you should be excited about ClojureScriptWhy you should be excited about ClojureScript
Why you should be excited about ClojureScriptelliando dias
 
Functional Programming with Immutable Data Structures
Functional Programming with Immutable Data StructuresFunctional Programming with Immutable Data Structures
Functional Programming with Immutable Data Structureselliando dias
 
Nomenclatura e peças de container
Nomenclatura  e peças de containerNomenclatura  e peças de container
Nomenclatura e peças de containerelliando dias
 
Polyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better AgilityPolyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better Agilityelliando dias
 
Javascript Libraries
Javascript LibrariesJavascript Libraries
Javascript Librarieselliando dias
 
How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!elliando dias
 
A Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the WebA Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the Webelliando dias
 
Introdução ao Arduino
Introdução ao ArduinoIntrodução ao Arduino
Introdução ao Arduinoelliando dias
 
Incanter Data Sorcery
Incanter Data SorceryIncanter Data Sorcery
Incanter Data Sorceryelliando dias
 
Fab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine DesignFab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine Designelliando dias
 
The Digital Revolution: Machines that makes
The Digital Revolution: Machines that makesThe Digital Revolution: Machines that makes
The Digital Revolution: Machines that makeselliando dias
 
Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.elliando dias
 
Hadoop and Hive Development at Facebook
Hadoop and Hive Development at FacebookHadoop and Hive Development at Facebook
Hadoop and Hive Development at Facebookelliando dias
 
Multi-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case StudyMulti-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case Studyelliando dias
 

More from elliando dias (20)

Clojurescript slides
Clojurescript slidesClojurescript slides
Clojurescript slides
 
Why you should be excited about ClojureScript
Why you should be excited about ClojureScriptWhy you should be excited about ClojureScript
Why you should be excited about ClojureScript
 
Functional Programming with Immutable Data Structures
Functional Programming with Immutable Data StructuresFunctional Programming with Immutable Data Structures
Functional Programming with Immutable Data Structures
 
Nomenclatura e peças de container
Nomenclatura  e peças de containerNomenclatura  e peças de container
Nomenclatura e peças de container
 
Geometria Projetiva
Geometria ProjetivaGeometria Projetiva
Geometria Projetiva
 
Polyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better AgilityPolyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better Agility
 
Javascript Libraries
Javascript LibrariesJavascript Libraries
Javascript Libraries
 
How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!
 
Ragel talk
Ragel talkRagel talk
Ragel talk
 
A Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the WebA Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the Web
 
Introdução ao Arduino
Introdução ao ArduinoIntrodução ao Arduino
Introdução ao Arduino
 
Minicurso arduino
Minicurso arduinoMinicurso arduino
Minicurso arduino
 
Incanter Data Sorcery
Incanter Data SorceryIncanter Data Sorcery
Incanter Data Sorcery
 
Rango
RangoRango
Rango
 
Fab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine DesignFab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine Design
 
The Digital Revolution: Machines that makes
The Digital Revolution: Machines that makesThe Digital Revolution: Machines that makes
The Digital Revolution: Machines that makes
 
Hadoop + Clojure
Hadoop + ClojureHadoop + Clojure
Hadoop + Clojure
 
Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.
 
Hadoop and Hive Development at Facebook
Hadoop and Hive Development at FacebookHadoop and Hive Development at Facebook
Hadoop and Hive Development at Facebook
 
Multi-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case StudyMulti-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case Study
 

Recently uploaded

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 

Recently uploaded (20)

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 

Scaling with Postgres

  • 1. Scaling with Postgres Robert Treat Percona Performance Conference 2009 Wednesday, April 22, 2009
  • 2. Disclaimer Every “really big” site uses custom solutions. Don’t expect to get there using out of the box software. You don’t have to reinvent the wheel. But you may have to build the car. Wednesday, April 22, 2009
  • 3. Who am I ? Wednesday, April 22, 2009
  • 4. Who am I ? OmniTI TB+ databases top 100 websites Oracle/MySQL/Postgres Not necessarily in that order :-) Wednesday, April 22, 2009
  • 6. Scaling Databases 101 Implementation Wednesday, April 22, 2009
  • 7. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Wednesday, April 22, 2009
  • 8. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Wednesday, April 22, 2009
  • 9. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication Wednesday, April 22, 2009
  • 10. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication Federation Wednesday, April 22, 2009
  • 11. Implementation Normalization Foreign Keys Relational Modeling Wednesday, April 22, 2009
  • 12. Implementation Normalization Foreign Keys Relational Modeling Mostly Irrelevant Wednesday, April 22, 2009
  • 14. Implementation Monitoring nagios, munin, etc.. check_postgres Wednesday, April 22, 2009
  • 15. Implementation Monitoring nagios, munin, etc.. check_postgres Trending cacti, noit watch your hardware! Wednesday, April 22, 2009
  • 17. Tuning Get the Basics Right http://wiki.postgresql.org/wiki/ Tuning_Your_PostgreSQL_Server Wednesday, April 22, 2009
  • 18. Tuning Get the Basics Right http://wiki.postgresql.org/wiki/ Tuning_Your_PostgreSQL_Server Someday Xlogs, Checkpoints Vacuum Strategy Costs and other Minutia Wednesday, April 22, 2009
  • 19. Optimize Queries explain explain analyze explain analyze verbose Wednesday, April 22, 2009
  • 20. Optimize Queries writing queries tuned to your databases planner Wednesday, April 22, 2009
  • 22. Optimize Schema indexing partial, functional fix data types add/remove tables de-normalize (maybe) Wednesday, April 22, 2009
  • 23. Why is this important? Postgres is good at scaling vertically Flawed database schema hides the real problems Wednesday, April 22, 2009
  • 24. Why is this unimportant? Optimizing is often a trade off of time vs. money It is not an ultimate solution Wednesday, April 22, 2009
  • 25. Throw Hardware At It Find your bottlenecks Memory / RAM CPU I/O, DISKS usually the biggest problem Wednesday, April 22, 2009
  • 26. Can you take me higher? 32 Cores 128 GB+ RAM Spindles out the wazoo tablespaces Wednesday, April 22, 2009
  • 28. Caching Internal materialized views Wednesday, April 22, 2009
  • 29. Caching Internal materialized views External memcached, et al. static files Wednesday, April 22, 2009
  • 30. Oh, the places you’ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits Wednesday, April 22, 2009
  • 31. Oh, the places you’ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits Top 1000+ Website Wednesday, April 22, 2009
  • 32. Replication Ignore the hype Triggers won’t destroy your site Scripting solves a lot of config issues Telling your DBA to do his !@#$ job solves config issues too Add-on is FUD for Modular Wednesday, April 22, 2009
  • 33. Replication Truth in advertising Oracle does it better MySQL does it simpler Wednesday, April 22, 2009
  • 35. Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII Wednesday, April 22, 2009
  • 36. Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII Scaling Writes Mostly impossible Bucardo, PGPoolII Wednesday, April 22, 2009
  • 37. What is Federation? A process for eliminating performance and/or scaleability as a requirement for your database Wednesday, April 22, 2009
  • 39. Federation MySQL software has no federation advantage Wednesday, April 22, 2009
  • 40. Federation MySQL software has no federation advantage MySQL community has large federation advantage Wednesday, April 22, 2009
  • 41. Federation Things to consider Horizontal Partitioning Application Based Partitioning BASE golconde, gearman Wednesday, April 22, 2009
  • 42. El Fin www.xzilla.net www.omniti.com www.postgresql.us Wednesday, April 22, 2009
  • 43. Postgres BOF Tonight @ 7:30 PM Location @ Camino Real Wednesday, April 22, 2009