9. SELECT "postgres_settings".* FROM "postgres_settings" WHERE
"postgres_settings"."database_id" = $1 AND
"postgres_settings"."invalidated_at_snapshot_id" IS NULL AND (id
not in
(70288,70289,70290,70291,70292,70293,70294,70295,70296,70297,70298
,70299,70300,70301,70302,70303,70304,70305,70306,70307,70308,70309
,70310,70311,70312,70313,70314,70315,70316,70317,70318,70319,70320
,70321,70322,70323,70324,70325,70326,70327,99059,99060,70330,70331
,70332,70333,70334,70335,70336,70337,70338,99061,70340,70341,70342
,70343,70344,70345,70346,70347,70348,70349,70350,70351,70352,70353
,70354,70355,70356,70357,70358,70359,70360,99062,70362,70363,70364
,70365,70366,70367,70368,70369,70370,70371,70372,70373,70374,70375
,70376,70377,70378,70379,70380,70381,70382,70383,70384,70385,70386
,99063,99064,99065,99066,99067,70392,70393,70394,70395,70396,70397
,70398,70399,70400,70401,70402,70403,70404,70405,99068,70407,70408
,70409,70410,70411,70412,70413,70414,70415,70416,70417,99069,70419
,70420,70421,99070,70423,70424,70425,70426,70427,70428,
Truncation
Improving
Data Quality
10. Improving
Data Quality
-[ RECORD 1 ]———+--------------------------------
query | SELECT * FROM x WHERE y = ?
calls | 5
total_time | 15.249
-[ RECORD 2 ]———+--------------------------------
query | SELECT * FROM z WHERE a = 123
calls | 50
total_time | 104.19
Race Condition during
pg_stat_statements_reset()
12. Fingerprinting
SELECT a AS b == SELECT a AS c
Problematic:
y IN (?, ?, ?) != y IN (?, ?)
Improving
Data Quality
SELECT a, b FROM x != SELECT b, a FROM x
DEALLOCATE p141 != DEALLOCATE p150
14. pg_stat_plans
Improving
Data Quality
pg_stat_statements variant that
differentiates between query plans.
Slower + Don’t use it before this bug is fixed:
https://github.com/2ndQuadrant/pg_stat_plans/issues/39
32. EXPLAIN SELECT * FROM x WHERE y = 1
QUERY PLAN
---------------------------------------------------------------------
Index Scan using idx_for_y on x (cost=0.15..8.17 rows=1 width=140)
Index Cond: (id = 1)
Parse Analyze Plan
pg_query
33. EXPLAIN SELECT * FROM x WHERE y = ?
ERROR: syntax error at or near ";"
LINE 1: EXPLAIN SELECT * FROM x WHERE y = ?;
Parse Analyze Plan
pg_query
34. EXPLAIN SELECT * FROM x WHERE y = ?
EXPLAIN SELECT * FROM x WHERE y = $1
ERROR: there is no parameter $1
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $1;
Parse Analyze Plan
pg_query
46. monitor.rb
Filtering &
Regression
Testing
Simple top-like tool that shows
pg_stat_statements data
https://gist.github.com/lfittl/301542602607b738b23f
47. Filtering &
Regression
Testing
monitor.rb -d testdb
AVG | QUERY
--------------------------------------------------------------------------------
10.7ms | SELECT oid, typname, typelem, typdelim, typinput FROM pg_type
3.0ms | SET time zone 'UTC'
0.4ms | SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid),
a.attnotnull, a.atttypid, a.atttypmod FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid
= d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = ?::regclass AND a.attnum > ? AND NOT
a.attisdropped ORDER BY a.attnum
0.2ms | SELECT pg_stat_statements_reset()
0.1ms | SELECT query, calls, total_time FROM pg_stat_statements
0.1ms | SELECT attr.attname FROM pg_attribute attr INNER JOIN pg_constraint cons ON attr.attrelid
= cons.conrelid AND attr.attnum = cons.conkey[?] WHERE cons.contype = ? AND cons.conrelid = ?:
:regclass
0.0ms | SELECT COUNT(*) FROM pg_class c LEFT JOIN pg_namespace n ON n.oid = c.relnamespace WHERE
c.relkind in (?,?) AND c.relname = ? AND n.nspname = ANY (current_schemas(?))
0.0ms | SELECT * FROM posts JOIN users ON (posts.author_id = users.id) WHERE users.login = ?;
0.0ms | SET client_min_messages TO 'panic'
0.0ms | set client_encoding to 'UTF8'
0.0ms | SHOW client_min_messages
0.0ms | SELECT * FROM ad_reels WHERE id = ?;
0.0ms | SELECT * FROM posts WHERE guid = ?;
0.0ms | SELECT ?
0.0ms | SET client_min_messages TO 'warning'
0.0ms | SET standard_conforming_strings = on
0.0ms | SELECT "posts".* FROM "posts" ORDER BY "posts"."id" DESC LIMIT ?
0.0ms | SHOW TIME ZONE
48. Filtering &
Regression
Testing
monitor.rb -d testdb -t posts
AVG | QUERY
--------------------------------------------------------------------------------
0.0ms | SELECT * FROM posts JOIN users ON (posts.author_id = users.id) WHERE users.login = ?;
0.0ms | SELECT * FROM posts WHERE guid = ?;
0.0ms | SELECT "posts".* FROM "posts" ORDER BY "posts"."id" DESC LIMIT ?
49. Filtering &
Regression
Testing
if cli.config[:table]
q = PgQuery.parse(query["query"])
next unless q.tables.include?(cli.config[:table])
end
51. Which query plans are affected by
removal of an index?
!
How would execution plans be
affected by an upgrade to 9.X?
Filtering &
Regression
Testing
52. Regression Test based on
pg_stat_statements
+ table statistics.
!
(no actual data)
Filtering &
Regression
Testing
53. Schema Dump +
Table Level Statistics
"n_live_tup": 75,
"relpages": 1,
"reltuples": 75.0,
“stanumbers1": [..],
"stavalues1": “{..}”,
…
Local Test
Database
Testing Setup
Production
Database
EXPLAIN SELECT FROM x WHERE y = ?
Filtering &
Regression
Testing
54. EXPLAIN SELECT * FROM x WHERE y = ?
EXPLAIN SELECT * FROM x WHERE y = $1
ERROR: there is no parameter $1
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $1;
Parse Analyze Plan
Filtering &
Regression
Testing
55. y = $1
ERROR: there is no parameter $0
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $0;
Filtering &
Regression
Testing
56. y = $1
ERROR: there is no parameter $0
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $0;
y = NULL
QUERY PLAN
----------------------------------------------------------------
Result (cost=0.00..21.60 rows=1 width=40)
One-Time Filter: NULL::boolean
-> Seq Scan on x (cost=0.00..21.60 rows=1 width=40)
Filtering &
Regression
Testing
57. y = $1
ERROR: there is no parameter $0
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $0;
y = NULL
QUERY PLAN
----------------------------------------------------------------
Result (cost=0.00..21.60 rows=1 width=40)
One-Time Filter: NULL::boolean
-> Seq Scan on x (cost=0.00..21.60 rows=1 width=40)
y = (SELECT null)
ERROR: failed to find conversion function from unknown to integer
Filtering &
Regression
Testing
58. y = $1
ERROR: there is no parameter $0
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $0;
y = NULL
QUERY PLAN
----------------------------------------------------------------
Result (cost=0.00..21.60 rows=1 width=40)
One-Time Filter: NULL::boolean
-> Seq Scan on x (cost=0.00..21.60 rows=1 width=40)
y = (SELECT null)
ERROR: failed to find conversion function from unknown to integer
y = (SELECT null::integer)
QUERY PLAN
----------------------------------------------------------------------
Index Scan using idx_for_y on x (cost=0.16..8.18 rows=1 width=144)
Index Cond: (y = $0)
InitPlan 1 (returns $0)
-> Result (cost=0.00..0.01 rows=1 width=0)
Filtering &
Regression
Testing
59. Finding out the type
y = $1
ERROR: there is no parameter $1
LINE 1: EXPLAIN SELECT * FROM x WHERE y = $1;
pg_prepared_statements
PREPARE tmp AS SELECT * FROM x WHERE y = $1;
SELECT unnest(parameter_types) AS data_type
FROM pg_prepared_statements WHERE name = ‘tmp’;
DEALLOCATE tmp;
data_type
-----------
integer
Filtering &
Regression
Testing
60. EXPLAIN SELECT * FROM x WHERE y = ?
EXPLAIN SELECT * FROM x WHERE y = $0
EXPLAIN SELECT * FROM x WHERE y =
((SELECT null::integer)::integer)
QUERY PLAN
---------------------------------------------------------------------
Index Scan using idx_for_y on x (cost=0.16..8.18 rows=1 width=144)
Index Cond: (y = $0)
InitPlan 1 (returns $0)
-> Result (cost=0.00..0.01 rows=1 width=0)
Parse Analyze Plan
Filtering &
Regression
Testing
61. Open Issue:
Planner reads actual
physical size whilst planning
Filtering &
Regression
Testing