O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Overview of Postgres 9.5

2.426 visualizações

Publicada em

This presentation reviews the top ten new features that will appear in the Postgres 9.5 release.

Postgres 9.5 adds many features designed to enhance the productivity of developers: UPSERT, CUBE, ROLLUP, JSONB functions, and PostGIS improvements. For administrators, it has row-level security, a new index type, and performance enhancements for large servers.

Publicada em: Software
  • Seja o primeiro a comentar

Overview of Postgres 9.5

  1. 1. Major Features: Postgres 9.5 BRUCE MOMJIAN POSTGRESQL is an open-source, full-featured relational database. This presentation gives an overview of the Postgres 9.5 release. Creative Commons Attribution License http://momjian.us/presentations Last updated: November, 2015 1 / 40
  2. 2. PostgreSQL the database… ◮ Open Source Object Relational DBMS since 1996 ◮ Distributed under the PostgreSQL License ◮ Similar technical heritage as Oracle, SQL Server & DB2 ◮ However, a strong adherence to standards (ANSI-SQL 2008) ◮ Highly extensible and adaptable design ◮ Languages, indexing, data types, etc. ◮ E.g. PostGIS, JSONB, SQL/MED ◮ Extensive use throughout the world for applications and organizations of all types ◮ Bundled into Red Hat Enterprise Linux, Ubuntu, CentOS and Amazon Linux Major Features: Postgres 9.5 2 / 40
  3. 3. PostgreSQL the community… ◮ Independent community led by a Core Team of six ◮ Large, active and vibrant community ◮ www.postgresql.org ◮ Downloads, Mailing lists, Documentation ◮ Sponsors sampler: ◮ Google, Red Hat, VMWare, Skype, Salesforce, HP and EnterpriseDB ◮ http://www.postgresql.org/community/ Major Features: Postgres 9.5 3 / 40
  4. 4. EnterpriseDB the company… ◮ Worldwide leader of Postgres based products and services founded in 2004 ◮ Customers include 63 of the Fortune 500 and 137 of the Forbes Global 2000 ◮ 2,700 customers range across many industries including financial, healthcare, government, retail, telcommunications, and insurance ◮ Enterprise offerings: ◮ PostgreSQL Support, Services and Training ◮ Postgres Plus Advanced Server with Oracle Compatibility ◮ Tools for Monitoring, Replication, HA, Backup & Recovery Community ◮ Citizenship ◮ Contributor of key features: Materialized Views, JSON, & more ◮ Nine community members on staff Major Features: Postgres 9.5 4 / 40
  5. 5. EnterpriseDB the company… Major Features: Postgres 9.5 5 / 40
  6. 6. EnterpriseDB Is a Leader The Gartner report, Magic Quadrant for Operational Database Management Systems, by Donald Feinberg, Merv Adrian, Nick Heudecker, Adam Ronthal, and Terilyn Palanca was published October 12, 2015. Major Features: Postgres 9.5 6 / 40
  7. 7. 9.5 Feature Outline 1. INSERT … ON CONFLICT, also known as “UPSERT” 2. Block-Range Indexes (BRIN) which enable compact indexing of very large tables 3. Analytic operations GROUPING SETS, CUBE, and ROLLUP 4. Row-Level Security (RLS) 5. In-memory sorting and hashing performance improvements 6. Multi-core and large memory scalability improvements 7. Automated management of the number of WAL files 8. Additional JSONB data manipulation functions and operators 9. Enhancements to Foreign Data Wrappers 10. Allow Indexed PostGIS LIMIT distance calculations without CTEs To be released in 2015, full item list at http://www.postgresql.org/docs/devel/static/release-9-5.html Major Features: Postgres 9.5 7 / 40
  8. 8. 1. INSERT … ON CONFLICT ◮ Turns a conflicting INSERT into an UPDATE ◮ Works for VALUES and SELECT as a row source ◮ Handles concurrent operations without errors ◮ Is row-oriented, unlike MERGE, which is batch-oriented ◮ Does not have the problems associated with the UPSERT/MERGE implementations of other vendors (http://www.pgcon.org/2014/schedule/attachments/327_upsert_weird.pdf) Major Features: Postgres 9.5 8 / 40
  9. 9. INSERT … ON CONFLICT Example CREATE TABLE ins_update_test (x INTEGER PRIMARY KEY); INSERT INTO ins_update_test VALUES (1); INSERT INTO ins_update_test VALUES (1); ERROR: duplicate key value violates unique constraint "ins_update_test_pkey" DETAIL: Key (x)=(1) already exists. Major Features: Postgres 9.5 9 / 40
  10. 10. INSERT … ON CONFLICT Example INSERT INTO ins_update_test VALUES (1) ON CONFLICT DO NOTHING; INSERT 0 0 INSERT INTO ins_update_test VALUES (1) ON CONFLICT (x) DO UPDATE SET x = 2; INSERT 0 1 SELECT * FROM ins_update_test; x --- 2 Major Features: Postgres 9.5 10 / 40
  11. 11. INSERT … ON CONFLICT … EXCLUDED Example CREATE TABLE customer (cust_id INTEGER PRIMARY KEY, name TEXT); INSERT INTO customer VALUES (100, ’Big customer’); INSERT INTO customer VALUES (100, ’Non-paying customer’); ERROR: duplicate key value violates unique constraint "customer_pkey" DETAIL: Key (cust_id)=(100) already exists. INSERT INTO customer VALUES (100, ’Non-paying customer’) ON CONFLICT (cust_id) DO UPDATE SET name = EXCLUDED.name; SELECT * FROM customer; cust_id | name ---------+--------------------- 100 | Non-paying customer Major Features: Postgres 9.5 11 / 40
  12. 12. INSERT … ON CONFLICT with SELECT CREATE TABLE merge (x INTEGER PRIMARY KEY); INSERT INTO merge VALUES (1), (3), (5); INSERT INTO merge SELECT * FROM generate_series(1, 5); ERROR: duplicate key value violates unique constraint "merge_pkey" DETAIL: Key (x)=(1) already exists Major Features: Postgres 9.5 12 / 40
  13. 13. INSERT … ON CONFLICT with SELECT INSERT INTO merge SELECT * FROM generate_series(1, 5) ON CONFLICT DO NOTHING; SELECT * FROM merge; x --- 1 3 5 2 4 Major Features: Postgres 9.5 13 / 40
  14. 14. INSERT … ON CONFLICT … UPDATE with SELECT CREATE TABLE merge2 (x INTEGER PRIMARY KEY, status TEXT); INSERT INTO merge2 VALUES (1, ’old’), (3, ’old’), (5, ’old’); INSERT INTO merge2 SELECT *, ’new’ FROM generate_series(2, 5) ON CONFLICT (x) DO UPDATE SET status = ’conflict’; SELECT * FROM merge2; x | status ---+---------- 1 | old 2 | new 3 | conflict 4 | new 5 | conflict Major Features: Postgres 9.5 14 / 40
  15. 15. 2. Block-Range Indexes (BRIN) ◮ Tiny indexes designed for large tables ◮ Minimum/maximum values stored for a range of blocks (default 1MB, 128 8k pages) ◮ Allows skipping large sections of the table that cannot contain matching values ◮ Ideally for naturally-ordered tables, e.g. insert-only tables are chronologically ordered ◮ Index is 0.003% the size of the heap ◮ Indexes are inexpensive to update ◮ Index every column at little cost ◮ Slower lookups than btree Major Features: Postgres 9.5 15 / 40
  16. 16. Block-Range Indexes (BRIN) Example CREATE TABLE brin_example AS SELECT generate_series(1,100000000) AS id; CREATE INDEX btree_index ON brin_example(id); CREATE INDEX brin_index ON brin_example USING brin(id); SELECT relname, pg_size_pretty(pg_relation_size(oid)) FROM pg_class WHERE relname LIKE ’brin_%’ OR relname = ’btree_index’ ORDER BY relname; relname | pg_size_pretty --------------+---------------- brin_example | 3457 MB btree_index | 2142 MB brin_index | 104 kB http://michael.otacoo.com/postgresql-2/postgres-9-5-feature-highl Major Features: Postgres 9.5 16 / 40
  17. 17. 3. Analytic Operations GROUPING SETS, CUBE, and ROLLUP ◮ Allows specification of multiple GROUP BY combinations in a single query ◮ Avoids the need for UNION ALL and recomputation ◮ Empty fields are left NULL Major Features: Postgres 9.5 17 / 40
  18. 18. Employee Table SELECT * FROM employee ORDER BY name; name | office | department -------+--------+------------ Jill | PHL | Marketing Lilly | SFO | Sales Mark | PHL | Marketing Nancy | PHL | Sales Sam | SFO | Sales Tim | PHL | Shipping Major Features: Postgres 9.5 18 / 40
  19. 19. GROUP BY Example SELECT office, COUNT(*) FROM employee GROUP BY office; office | count --------+------- SFO | 2 PHL | 4 SELECT department, COUNT(*) FROM employee GROUP BY department; department | count ------------+------- Marketing | 2 Shipping | 1 Sales | 3 Major Features: Postgres 9.5 19 / 40
  20. 20. GROUP BY with UNION ALL SELECT office, NULL as department, COUNT(*) FROM employee GROUP BY office UNION ALL SELECT NULL as office, department, COUNT(*) FROM employee GROUP BY department ORDER BY 1; office | department | count --------+------------+------- PHL | | 4 SFO | | 2 | Marketing | 2 | Shipping | 1 | Sales | 3 Major Features: Postgres 9.5 20 / 40
  21. 21. GROUPING SETS Example SELECT office, department, COUNT(*) FROM employee GROUP BY GROUPING SETS (office, department) ORDER BY office, department; office | department | count --------+------------+------- PHL | | 4 SFO | | 2 | Marketing | 2 | Sales | 3 | Shipping | 1 Major Features: Postgres 9.5 21 / 40
  22. 22. ROLLUP Example SELECT office, department, COUNT(*) FROM employee GROUP BY ROLLUP (office, department) ORDER BY office, department; office | department | count --------+------------+------- PHL | Marketing | 2 PHL | Sales | 1 PHL | Shipping | 1 PHL | | 4 SFO | Sales | 2 SFO | | 2 | | 6 Major Features: Postgres 9.5 22 / 40
  23. 23. CUBE Example SELECT office, department, COUNT(*) FROM employee GROUP BY CUBE (office, department) ORDER BY office, department; office | department | count --------+------------+------- PHL | Marketing | 2 PHL | Sales | 1 PHL | Shipping | 1 PHL | | 4 SFO | Sales | 2 SFO | | 2 | Marketing | 2 | Sales | 3 | Shipping | 1 | | 6 Major Features: Postgres 9.5 23 / 40
  24. 24. GROUPING SETS Equivalent of CUBE SELECT office, department, COUNT(*) FROM employee GROUP BY GROUPING SETS ((office, department), office, department, ()) ORDER BY office, department; office | department | count --------+------------+------- PHL | Marketing | 2 PHL | Sales | 1 PHL | Shipping | 1 PHL | | 4 SFO | Sales | 2 SFO | | 2 | Marketing | 2 | Sales | 3 | Shipping | 1 | | 6 Major Features: Postgres 9.5 24 / 40
  25. 25. 4. Row-Level Security (RLS) ◮ Allows SELECT, INSERT, UPDATE, OR DELETE permission control over existing rows with USING expression ◮ Also INSERT or UPDATE control over added and modified rows with CHECK expression ◮ Expressions can contain checks for the current user, subqueries, time comparisons, and function calls ◮ Enabled with GUC row_security, CREATE POLICY, and ALTER TABLE … ENABLE ROW LEVEL SECURITY Major Features: Postgres 9.5 25 / 40
  26. 26. Row-Level Security Example Table Setup SHOW row_security; row_security -------------- on CREATE TABLE orders (id INTEGER, product TEXT, entered_by TEXT); ALTER TABLE orders ENABLE ROW LEVEL SECURITY; CREATE POLICY orders_control ON orders FOR ALL TO PUBLIC USING (entered_by = CURRENT_USER); GRANT ALL ON TABLE orders TO PUBLIC; Major Features: Postgres 9.5 26 / 40
  27. 27. Row-Level Security Example User Setup CREATE USER emp1; CREATE USER emp2; SET SESSION AUTHORIZATION emp1; INSERT INTO orders VALUES (101, ’fuse’, CURRENT_USER); SET SESSION AUTHORIZATION emp2; INSERT INTO orders VALUES (102, ’bolt’, CURRENT_USER); Major Features: Postgres 9.5 27 / 40
  28. 28. Row-Level Security Example Testing SET SESSION AUTHORIZATION postgres; SELECT * FROM orders; id | product | entered_by -----+---------+------------ 101 | fuse | emp1 102 | bolt | emp2 Major Features: Postgres 9.5 28 / 40
  29. 29. Row-Level Security Example Testing SET SESSION AUTHORIZATION emp1; SELECT * FROM orders; id | product | entered_by -----+---------+------------ 101 | fuse | emp1 SET SESSION AUTHORIZATION emp2; SELECT * FROM orders; id | product | entered_by -----+---------+------------ 102 | bolt | emp2 Major Features: Postgres 9.5 29 / 40
  30. 30. 5. In-Memory Sorting and Hashing Performance Improvements ◮ Allow VARCHAR(), TEXT and NUMERIC() to use the abbreviated sorting optimization ◮ Use memcmp()as quick string equality checks before collation comparisons ◮ Decrease the average number of hash entries per bucket from 10 to 1 ◮ Pre-allocate the maximum number of hash buckets in cases where we are likely to use multiple work_mem-sized batches ◮ Allow CREATE INDEX, REINDEX, and CLUSTER to use inlined sorting ◮ Allow use of 128-bit accumulators for aggregate computations Major Features: Postgres 9.5 30 / 40
  31. 31. 6. Multi-Core and Large Memory Scalability Improvements ◮ Improve concurrency of shared buffer replacement ◮ Reduce the number of page locks and pins during index scans ◮ Make backend local tracking of buffer pins memory efficient ◮ Improve lock scalability on multi-socket systems ◮ Increase the number of shared buffer mapping hash table entries from 16 to 128 ◮ Allow searching for a free shared buffer to use minimal locking ◮ Force buffer descriptors to be CPU-cache aligned (128 bytes) ◮ Reduce btree page pinning Major Features: Postgres 9.5 31 / 40
  32. 32. 7. Automated Management of the Number of WAL Files ◮ New GUC variables min_wal_size and max_wal_size control the minimum and maximum size of the pg_xlog directory ◮ Previously checkpoint_segments controlled only the maximum directory size (previously WAL files were not removed) ◮ Size specified in bytes, not segment files ◮ Allows use of additional WAL files only when needed Major Features: Postgres 9.5 32 / 40
  33. 33. Management of WAL Files 000000000000 111111111111 000000000 111111111 000000000000 111111111111 000000 000 111111 111 00000000 0000 11111111 1111 00000000 0000 11111111 1111 000000000 111111111 000000000000 111111111111 000000000000 111111111111 000000000000 111111111111 000000000 111111111 0000000000000000 1111111111111111 0000000000000000 1111111111111111 000000000000 111111111111 111 1 1Begin 1 Rotate 2 22 2 2 2End 1 2 2 21 1 PostgreSQL Shared Buffer Cache Write−Ahead Log Major Features: Postgres 9.5 33 / 40
  34. 34. 8. Additional JSONB Data Manipulation Functions and Operators ◮ Add jsonb_set(), which allows replacement of or addition to JSONB documents ◮ Allow removal of JSONB documents using the subtraction operator ◮ Allow merging of JSONB documents using the concatenation (|| operator) ◮ Add function to remove null values from documents Major Features: Postgres 9.5 34 / 40
  35. 35. 9. Enhancements to Foreign Data Wrappers ◮ Add IMPORT FOREIGN SCHEMA to create a local table matching the schema of a foreign table ◮ Allow foreign tables to be part of inheritance trees ◮ Allow CHECK constraints on foreign tables ◮ Add infrastructure for foreign table join pushdown Major Features: Postgres 9.5 35 / 40
  36. 36. 10. Allow Indexed PostGIS LIMIT Distance Calculations without CTEs ◮ Nearest neighbor searches allow index lookups to return the closest matches, e.g. return the 10 nearest points to a given point ◮ Only the bounding boxes of two-dimensional objects are indexed, e.g. polygon, circle, line ◮ Previously LIMIT could not combine bounding box index lookups with accurate calculations ◮ Now LIMIT bounding box index filtering can recheck using accurate distance calculations ◮ Workaround was to use a CTE with a 10x limit, then an outer query to do accurate distance calculations Major Features: Postgres 9.5 36 / 40
  37. 37. Pre-9.5 LIMIT Distance Example WITH index_query AS ( SELECT st_distance(geom, ’SRID=3005;POINT(1011102 450541)’) AS distance, parcel_id, address FROM parcels ORDER BY geom <-> ’SRID=3005;POINT(1011102 450541)’ LIMIT 100 ) SELECT * FROM index_query ORDER BY distance LIMIT 10; http://boundlessgeo.com/2011/09/indexed-nearest-neighbour-search-in-postgis/ http://shisaa.jp/postset/postgis-postgresqls-spatial-partner-part-3.html Major Features: Postgres 9.5 37 / 40
  38. 38. 9.5 LIMIT Distance Example SELECT st_distance(geom, ’SRID=3005;POINT(1011102 450541)’) AS distance, parcel_id, address FROM parcels ORDER BY geom <-> ’SRID=3005;POINT(1011102 450541)’ LIMIT 10 http://www.postgresonline.com/journal/archives/350-PostGIS-2.2-leveraging-power-of-PostgreSQL-9.5.html http://postgis.net/docs/manual-dev/geometry_distance_knn.html Major Features: Postgres 9.5 38 / 40
  39. 39. Additional Resources… ◮ Postgres Downloads: ◮ www.enterprisedb.com/downloads ◮ Product and Services information: ◮ info@enterprisedb.com Major Features: Postgres 9.5 39 / 40
  40. 40. Conclusion http://momjian.us/presentations https://www.flickr.com/photos/thevlue/ Major Features: Postgres 9.5 40 / 40

×