More Related Content Similar to Technical Introduction to PostgreSQL and PPAS (20) Technical Introduction to PostgreSQL and PPAS1. A Technical introduction
to PostgreSQL and PPAS
Enterprise Class PostgreSQL Database from EDB
10/6/2014
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
2. • Introduction
• About Ashnik
• PostgreSQL and buzz it has created
• Postgres Plus Advanced Server
• Architecture
• Oracle Compatibility
• Performance Features
• Security Features
• High Availability Features
• DBA Tools
• User Stories
• What’s coming up in v9.3
• How to start adopting
Agenda
2
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
3. • Independent & Thriving Development Community for over 20 years
• Thousands of active deployments worldwide in public and private
sector organizations of all sizes
PostgreSQL and the Community
3
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
7. EnterpriseDB – The Company
7
• The Enterprise PostgreSQL company was founded in 2004,
first product GA in 2005
• 2,000+ customers across all market segments
• 70,000+ downloads/week of PostgreSQL and related products
• Enabling database consolidation using PostgreSQL and
advanced Oracle compatibility
• Saving customers millions through the power of open source
• Strong financial backing:
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Postgres Plus - Recognized
by Gartner’s Magic
Quadrant as a challenger
in the Database Market
8. The EnterpriseDB Advantage
8
• Products and Tools
• Advanced database server software
• Deep Oracle compatibility
• Bundled development and management tools
• Technical Support and Services
• Around the clock support
• Expert consulting
• Oracle migration services
• Remote management and monitoring
• Professional Training
• Learn PostgreSQL from the experts
• Web and on-site training
• Training for developers and DBAs
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
9. • Fast development cycles
• Thousands of developers
• Better code
• Lower cost
• 24/7 support
• Services and training
• Certification
• Indemnification
• Product strategy
Bringing the Advantage to Enterprises
9
Open Source
Software
Commercial
SoftwareEnterpriseDB
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
13. Database Limitations
13
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• Limitations are generally
defined by
• Operating System Limits
• Compile-Time Parameters
– Data Type Usage
General Database
Limitations
Limit Value
Maximum Database Size Unlimited
Maximum Table Size 32 TB
Maximum Row Size 1.6 TB
Maximum Field Size 1 GB (4TB in v9.3 Beta)
Maximum Rows per Table Unlimited
Maximum Columns per Table 250-1600 (Depending on
Column types)
Maximum Indexes per Table Unlimited
14. Oracle Compatibility
14
• Run applications written for Oracle
virtually unchanged
• No need to re-train Oracle DBAs and
developers
• Support for PL/SQL language and OCI
interoperability
• Replication for easy sharing of data
• Dramatic Cost Savings
• No Vendor Lock-in
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
15. Survey: Re-Use of Oracle DBA Skills
15
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
16. Compatibility Means
16
• SQL Extension support
• Decode, NVL, Substr, NVL2,replace,translate, table()
• Date/time functions: add_months, extract, next_day,months_between,trunc, current_date support
• Support for cascade option in drop table
• PL/SQL support
• REF Cursors, Implicit and explicit cursors
• Looping, variable declarations, conditional statements
• Collections: Associative Arrays, Varrays, Nested tables
• Bulk binding
• Named parameters
• User Defined Exceptions
• Explicit Transaction Control - within a stored procedure
• Tools
• EDB*Plus – SQL*Plus look-a-like
• EDB*Loader – SQL*Loader equivalent
• EDB*Wrap – similar to the PL/SQL wrapper
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
17. Compatibility Means (cont.)
17
• Packages
• Stored procedures
• Functions
• Triggers
• Optimizer Hints
• Database Links
• Hierarchical Queries
• Synonyms – Public and
Private
• Sequences
• Materialized Views
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• Rownum
• Object types
o Create type … as object
o Create type … as table
o Create type …as varray
o Constructor and collection
methods
• PL/SQL like SUBTYPE, which
inherits form base type
• Users/Roles
• Dynamic SQL
• Features
18. Compatibility Means (cont.)
18
• Data Types
• Integer, number, char, double precision, float, varchar2, blob, clob, xmltype, rowid, boolean
• Built-in Packages
• DBMS_:
o SQL, LOB, JOB, PIPE, ALERT, OUTPUT, UTILITY, PROFILER, RLS
• UTL_:
o FILE, MAIL, SMTP, ENCODE, TCP
• Oracle-like Data Dictionary
• ALL_, DBA_, USER_ views
• Most commonly accessed views
• Diagnostics - DRITA
• System and session waits
o Not exposed in PostgreSQL
o Part of Advanced Server
• Statspack-like reporting
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
19. Flexible Partitioning Scheme in PostgreSQL
19
• Partitioning Example in PostgreSQL
o CREATE TABLE employees
o Create Child Tables
o create table emp_mgmt (check (job in (MANAGER'))) inherits (employees);
o create table emp_sales (check (job in ('SALESMAN'))) inherits (employees);
o Create the partitioning function
o CREATE OR REPLACE FUNCTION partition_emp();
o Create the trigger for partition
o CREATE TRIGGER employees_partition_trigger BEFORE INSERT ON employees
FOR EACH ROW EXECUTE PROCEDURE partition_emp();
• Advantages
• Flexible
• Customize your Partitioning Policy
• Disadvantages
• Complex
• Difficult to add/delete/split Partitions
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
20. • Partitioning Example in PPAS
CREATE TABLE employees
(empno numeric(4,0),ename varchar(10),job varchar(9),hiredate timestamp)
PARTITION BY LIST (job)
(PARTITION emp_mgmt VALUES ('MANAGER', 'PRESIDENT') tablespace tbsp_1, PARTITION
emp_sales VALUES ('SALESMAN') tablespace tbsp_2, PARTITION emp_ops VALUES ('CLERK')
tablespace tbsp_3
);
• PPAS Offers
• LIST and RANGE Partitioning Syntax
• Easy to manage and maintain partitions
• You can still take benefit of PostgreSQL syntax for complex partitioning e.g. Dynamic Partitions
• Easy to Add/Delete/Split/Swap/Detach/Attach partitions
• Brings the best of both the world!
Compatible and Flexible Partitioning Scheme in PPAS
20
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
21. Scaling with Partitioning in PPAS v9.3
21
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
100
250
500
1000
Time in Seconds
NumberofTablePartitions
Partitioning Performance: PPAS 9.3 v. PostgreSQL 9.3 v. PPAS 9.2 • Postgres Plus
Advanced Server 9.3
is up to 460 times
faster with row
insertions into
partitioned tables
than PostgreSQL 9.3
or PPAS 9.2
• Better performance
for bulk loading and
disaster recovery
22. • Online Migration Toolkit enables point and click migration from Oracle
• Automatically Migrates:
Database Migration Toolkit
22
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• Data
• Schemas
• Stored Procedures
• Triggers
• Functions
• Sequences
• Packages
• Views
• Database Links
• Synonyms
23. • Developed for High Performance Transaction
Environments (OLTP)
• DynaTune:
• Dynamic tuning of the database server to make the
optimal usage of the system resources available on
the host machine
• Index Advisor:
• Helps determine which columns you should index
to improve performance in a given workload.
• Query Optimization Hints, Hi-Speed Bulk Loader,
Multi-Threaded Replication
Performance enhancement in PPAS
23
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
24. • Infinite Cache
• High performance
horizontal scaling
architecture for
cache memory
• Cache expands with
inexpensive
commodity
hardware
Scalability with Infinite Cache
24
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
25. Scalability with Infinite Cache
25
• Architecture: How it Works
• Disk access for data is slower than RAM access
• Infinite Cache puts disk data into the RAM cache
• Additional requests for that data are read from the faster cache
• The cache scales as needed with inexpensive hardware
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Application
High
speed
pg_cach
e buffers
in RAM
Slow speed
disk
High
Speed
Network
Update
d Data
Blocks
Update
Read
If not in buffer
Database Server
InfiniteCache
RAMblades
High
speed
26. Scalability with Infinite Cache
26
• Single Machine Performance
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Advanced Server is 16X
faster on a single machine
with large amounts of
memory
(e.g. greater than 2 GB)
Infinite Cache can be used
on a single machine!
27. Scalability with Infinite Cache
27
• Infinite Cache - Features
• Expands and Distributes buffer cache across multiple machines
• Designed for Read-Mostly applications (e.g. Content Management, Query Intensive, Business
Intelligence, Data Warehouse)
• Cache is transparent to client applications (no cache coding needed)
• Compression feature enables caching entire databases (e.g. put a 250 GB database into 32 GB
RAM Cache)
• Cache can be pre-warmed for immediate results
• Cache scales using inexpensive commodity hardware
• Infinite Cache can be run to boost single machine performance!
• Created For
• DBAs and data managers overseeing large amounts of data requiring fast response times for
queries and reporting loads
• Developers who don’t want to write specialized caching code16
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
28. • Red Hat Reference Architecture Series:
Comparing BenchmarkSQL Performance
on Red Hat Enterprise Linux 5 to Windows
Server Enterprise (2010)
Performance Benchmark to SQL Server
28
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
30. Security
30
• Object level privileges assigned to roles and users
• Virtual Private Database
• Kerberos and LDAP authentication
• Host base authentication
• SSL communication
• Data Level Encryption (AES, 3DES, etc)
• Ability to utilize 3rd party Key Stores in a full PKI Infrastructure
• Foundation for full compliance with the strictest of security standards
(PCI Data Security Standard)
• Flexible field level encryption and row level security
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
“By default, PostgreSQL is probably the most
security-aware database available ...”
Database Hacker's Handbook- David Litchfield
31. PL/Secure and EDB*Wrap
31
• Encrypt your Pl/pgsql stored programs with PL/Secure
• Encrypt your SPL procedures with EDB*Wrap
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
32. SQL/Protect
32
• DBA Managed SQL Injection Protection
a) Preventing attacks is normally the responsibility of the application developer, but
with SQL/Protect, DBAs can now provide another layer of protection to prevent
corruption or co-opting of the database.
• Multiple Prevention Techniques
a) Unauthorized Relations
b) Utility Commands (e.g. DDL)
c) SQL Tautology
d) Unbounded DML
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
33. • Multi-Master Replication – Active-Active Technology
• Near real-time Replication
• Multi-Version Concurrency Control (MVCC)
• Point-in-Time Recovery
• Log Shipping for Standby (~ Oracle® Data Guard)
High Availability
33
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Availability
34. Backup and Recovery Options
34
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• Physical and Logical Backup
• Logical Backup using pg_dump
• Instance level logical backup using pg_dumpall
• Table level, Schema level or DB level logical backup
• pg_restore
• Physical Backups
• pg_basebackup
• Compressed backup
• Recovery with WAL and archived WAL
• Point in Time Recover
35. • Using OS Clustering
• Using a shared disk for data
Active – Passive OS HA Clustering
35
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
36. • Uses WAL (Write Ahead Log)
• WAL is continuously shipped through
an opened TCP channel to Hot
Standby Node
• Hot Standby node can be used for
read scalability
Hot Streaming Replication (Hot Standby Mode)
36
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
37. • Uses WAL (Write Ahead Log)
• Archived WAL is shipped to Hot
Standby Node
• Hot Standby node can be used for
read scalability
Log Shipping Replication (Hot Standby Mode)
37
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
39. • Using pg-pool
• Using redundant disk for data
• Hot Standby node can be used
for read scalability
HA with read scaling (with pg-pool)
39
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
40. • Trigger based replication
• Publication – Subscription
• Snapshot and Continuous
• Cascaded Replication
• Read Scalability
• Master/Slave DB can be:
• Oracle
• MSSQL
• Postgres Plus Advanced Server
• Either Master or Slave should be
PPAS/PostgreSQL
xDB Single Master Replication (SMR)
40
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
41. • High Availability uses
• Geographic distribution of load
• For creation of Testing/staging env using snapshot replication
• Segregate OLTP and reporting
xDB Replication Use Cases
41
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Replication
42. xDB Single-Master Replication (SMR)
42
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Table
D
Table
C
Table
B
Table
A
Table
C
Table
D
Procs
Objects
ReportsQueries
Continuous
or Scheduled
----------------
Filtered
or All Rows
Transaction
Replication
Improved
OLTP
Performance
Inexpensive
Query /
Reporting
Oracle Server Postgres Plus Advanced Server
43. xDB SMR Heterogeneous Replication Support
43
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Source/Target Oracle MS SQL
Server
2005/2008
PostgreSQL
Advanced
Server
(Oracle
mode)
Advanced
Server
(PostgreSQL
mode)
Oracle No No Yes Yes Yes
MS SQL
Server
2005/2008
No No Yes Yes Yes
PostgreSQL No Yes Yes Yes Yes
Advanced
Server (Oracle
mode)
Yes Yes No
Yes
No
Advanced
Server
(PostgreSQL
mode)
No Yes Yes Yes Yes
44. • Trigger based replication
• 2 or more Masters can be Sync
• Auto Conflict Detection & Resolution
• Read & Write Scalability
xDB Multi Master Replication (MMR)
44
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
45. • Java-based Replication Server and Replication Console
• Delta changes recorded in shadow tables via post Insert/Update/Delete triggers
• Data Replication through JDBC channel
xDB MMR Architecture
45
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
47. Remotely Manage, Monitor And Tune
• Management En-Mass Design
• Enterprise Performance Monitoring
• Proactive Alert Management
• Graphical Administration
• SQL Workload Profiling
• Simplified Capacity Planning
• Full SQL IDE
Postgres Enterprise Manager
47
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
49. • Performance
Monitoring Dashboards
• Capacity Manager
• Postgres Expert
• Alert Management
• Browser based console
and dashboard
• Audit Manager
• Team Support
• Distributed Architecture
• Convenient Access
PEM For DBAs: Centralized Tool
49
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
50. Helps you apply patches and
updates for all the
products under PPAS:
Stackbuilder Plus
50
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
51. • A solution to aid in the creation of highly
available configurations of Postgres
• Monitors the health of a Postgres HA
configuration
• Automates the failover process in the event
of a failure
• Used in conjunction with Streaming
Replication
Postgres Plus Failover Manager
51
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Slave
Witness
Master
53. • Automatic Failover from master to replica node
• Configurable fencing operation
1. By default uses VIP
2. Parameter to specify alternative operation
a) Ex: reconfigure a load balancer
• Manual failover configuration possible
• Email notifications when cluster status changes
• Witness node provides protection against ‘split brain’ scenarios
• User configurable wait times
• Built on PPCD/JGroups technology
Failover Manager Features
53
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
54. Replicate below objects from MS SQL Server to
Postgres Plus Advanced Server:
• Schemas
• Tables
• Constraints
• Indexes
• Table data
• Views
Replicating & Migrating Microsoft SQL Server
54
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
55. • Split input files into multiple files
• Load single tables in parallel
• Magnitudes better load time performance
• Parallel load capabilities also in Migration Toolkit and Migration Studio
Parallel Data Loader
55
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
56. FTD Replicates Oracle Data for Volume Peaks
Background
FTD is the world’s leading provider of floral-
related products and services
Processes 15M+ orders annually through 20,000
retail florists in the US
The ARGO System Project
FTD deployed new shipping administration
system (ARGO)
ARGO’s performance deteriorated during peak
holiday loads
Oracle-based system couldn’t handle peak
reporting volumes
Off-load reporting data form Oracle to Postgres
using Postgres Plus Replication Server
Why Postgres
No major changes required to run Oracle apps
on Postgres
Project completed in 6 weeks (Valentines Day
Mother’s Day)
Out of the Box Compatibility with existing
database
Improved Vendor Service – response times
decreased dramatically
Performance of order processing systems
improved 400% by offloading reporting to
Postgres
“When someone comes in at a sixth of the cost [of Oracle]
and 500 times the customer service, that makes it very
easy.”
Jason Weiss, Software Architect
57. InMobi Delivers Mobile Ads with Postgres
57
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Background
• One of the fastest growing mobile advertising networks
• Huge scalability and performance requirements, billions of ads per month
• Uses HP Proliant server
Mobile Ad Platform Project
• Postgres the best fit for a scalable open-source
database
• Community support not sufficient to meet business
requirements, EnterpriseDB a partner
• Initial database install supported organic growth
• 25 instances across 4 data centers with replication
• Database sizes upwards of 600 GB
• 200K txns/minute with 3 ms response time
Why Postgres
• Most scalable open-source database
• Lower cost than traditional databases
• Performance and reliability up to the task at
internet scale, supporting the most extreme
workloads
• Customizations by EnterpriseDB rapidly
incorporated into the product ensuring costly
customization maintenance is avoided
58. Sony Reduces TCO of database environment by 80%
58
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Free Realms Project
• Lower TCO and improve licensing flexibility
• Leverage existing Oracle DBA and Developer talent
• Migrate key Oracle apps to Postgres
• Commercial-grade quality and reliability, including
backup and recovery standards, to support mission-
critical applications
• Scalable, high performance execution
Why Postgres
• Reduced TCO by 80%
• East of Migration – majority of Oracle apps
could run unmodified on Postgres Plus
• Skills re-use – Oracle – trained staff could
immediately work with Postgres Plus
• Exceptional technical support from
EnterpriseDB
“Postgres Plus has proven to be a cost-effective
database that can accommodate Free Realms’
massive growth.”
- Andy Sites, Free Realms Senior Producer, SOE
Background
• Free Realms massively multiplayer online game
• >9M registered players since April 2009
• IT Oracle scale-up was cost-prohibitive
59. Background
• Multi-terabyte Oracle DB and needed to
reduce DB costs
• Running old, unsupported version of
Oracle and had to pay to upgrade
The EPBS Project
• Engaged EnterpriseDB consulting
services to migrate
• Unique migration process developed
and implemented
Why Postgres
• 2.2 TeraByte system migrated and
now running on Postgres Plus
Advanced Server
• Current performance is comparable
to Oracle
• Costs reduced significantly by
migrating to EnterpriseDB solutions
Achieved Comparable Performance Replacing Oracle
59
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
60. ToT, Thailand plans to replace Oracle Exadata-TimesTen In-
memory database with HP-EnterpriseDB stack
60
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Background
• Largest public sector company in telecom in Thailand
• Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries)
• Current setup has Oracle Exadata and four TimeTen In-memory databases
Current Challenges in Oracle Stack
• Issues in sync of Exadata with
In-memory databases
• Full refresh locks the tables
• Current system can go upto 1500 tps
• Dynamic Partitioning for huge table
• Current system load is 750 TPS with
write operations
• 5000 user requests (16.5K read queries)
per second with 4 servers
HP-EnterpriseDB Stack Proposed to be DR for Exadata
• HP DL-980 (4 CPU * 10 cores, 256GB RAM)
• 1TB VMA Disks for faster access
• Simulation tests show 2800 write-TPS achieved
• Flexible Partitioning scheme in PPAS (add partition on the fly for
each new date, no manual intervention)
• Currently this server is being tested as DR for Exadata
• Replication using xDB near-relatime replication for 1000 tps
• Scalability achieved using cascaded replication and Infinite Cache
• Setup can handle 5500 user requests (18K queries per second)
with 1.5-2.5ms response time per query
61. v.9.3
• Switch master without archive logs in cascaded replication
• Faster recovery
• Enhanced partitioning feature- Better performance
• More extended JSON support
• Materialized Views
• Updatable Views
• Recursive Views
• Replication between Heterogeneous System Architectures – Facilitates central backup server
• pg_xlogdump to decode WAL
• 4TB Large Object Support
• Better memory managements
What’s New?
61
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
62. v.9.4
• Logical Replication
• Time Lagging DR
• Enhanced JSON features
• Enhancements in Materialized Views
• Moving aggregates
• Pg_prewarm to warm up the cache
• Backup throttle
• ALTER SYSTEM command to set parameters
What’s Coming up?
62
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
63. How to start adoption
63
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
REPLICATION SERVER
BACKUP / DR SERVER
NEW APPLICATION
MIGRATION
64. You have a ‘Real’ alternative to Oracle or other
conventional proprietary Databases
Conclusion
64
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Economical
Technically
Sound
Easy to
Adopt