SlideShare uma empresa Scribd logo
1 de 34
Baixar para ler offline
LOCAL DEVELOPER DAY '09 /Winter
  (                @       , 2009 )
  http://www.local.gr.jp/?php%2Fhistory%2F%C2%E803%B2%F3%28LDD%2709%2FWinter%29




PostgreSQL 8.4 Update

     PostgreSQL                   / NTT OSS



                     2009.2.14
PostgreSQL 8.4

                                                       8.0
                                                                   8.3
                                                                                   8.4      2009      7
Ingress
 Ingress        PostgreSQL
                PostgreSQL                                        8.1
                    6.0
                    6.0                             ••
1977                                 7.4
                                     7.4            •2           (2PC)                   8.3
       1986
              1996
                                                     •2           (2PC)
                      2000                          ••                            •HOT:
                                                                                  •HOT:
                             2003
                                    2004                                          •
                                                                                  •
POSTGRES
POSTGRES                                     2005                                 •VACUUM
                                                                                  •VACUUM      //
                                                          2006
                     7.3
                     7.3                                                  2007
                                                                                 2008
                                       8.0
                             ••
                                                                                               2009
                             ••              (PITR)
                                              (PITR)
                             ••                                           8.2
                             •Windows                     •CPU
                                                          •CPU
                              •Windows
                                                          •
                                                          •
                                                          •
                                                          •
                                                          •GIN:
                                                          •GIN:
                                                                                               8.4

                                                                                                          2
8.4

           /

         HOT   XML       autovacuum
8.3
2008



                WITH     SQL
                                           pgtune
8.4            Window     Visibility Map
                                           pg_reorg
2009                 /




8.5
2010 ?


                                                      3
PostgreSQL 8.4 :        /               /



             /
                              WITH
                            Window
                                 /




      SQL                     pgtune
       Visibility Map        pg_reorg




                                            4
:                                     /

                    CHECK
             8.2
•         SELECT, UPDATE,
 DELETE
•

                                1         2   3           4
                                     4×           (       )
          8.4
  constraint_exclusion = partition
     CHECK

  EXECUTE USING in PL/pgSQL
                             INSERT

                                                              5
:       (EXECUTE USING)                /

                  INSERT
                                                : tbl
     CREATE TRIGGER BEFORE INSERT
        (                       )
                   CHECK
      CHECK('2009-01-01' <= insert_time
                                          1       2       3            4
        AND insert_time < '2009-02-01')                 : tbl_[   ][   ]

                     CREATE FUNCTION insert_trigger()
                       RETURNS TRIGGER AS
                     $$
                     DECLARE
                       part text;
     (8.3            BEGIN
IF            )        part := 'tbl_' ||
                               to_char(new.insert_time, 'YYYYMM');
                       EXECUTE 'INSERT INTO ' || part ||
                               ' VALUES(($1).*)' USING new;
                       RETURN NULL;
                     END;
                     $$                         EXECUTE USING
                     LANGUAGE plpgsql;              SQL new

                                                                           6
:                                         /

           effective_io_concurrency
                                I/O
RAID

                                      I/O
Bitmap Heap Scan
  ◎1      SQL                                 OLTP
  ×                                         OLAP




posix_fadvise(WILLNEED)
  POSIX     (      Linux) / Windows




                                                         7
/

    pg_restore
      -m, --multi-thread = number-of-threads


      PostgreSQL
            CPU




                                     PostgreSQL

            pg_dump
            --format=custom D
                                     pg_restore   R       R   R   R



(       )                                  (          )

                                                                      8
PostgreSQL 8.4 :


             /
                          WITH
                        Window
                             /




      SQL                 pgtune
       Visibility Map    pg_reorg




                                    9
WITH     (                    )
 WITH, WITH RECURSIVE
          ID
  =# TABLE tree;
   id | parent      1
  ----+--------
    1 |
    2 |       1   2 3                self-join
    3 |       1
    4 |       3       4
   ...
 =# WITH RECURSIVE r AS (
        SELECT * FROM tree WHERE id = 1
      UNION ALL
        SELECT tree.* FROM tree, r WHERE tree.parent = r.id
      )
    SELECT * FROM r ORDER BY id;
  id | parent
 ----+--------
   1 |
   2 |      1
   3 |      1
   4 |      3
 (4 rows)

                                                         10
WITH         (         ):
         ML                                        (!?)
WITH RECURSIVE Z(IX, IY, CX, CY, X, Y, I) AS (
               SELECT IX, IY, X::float, Y::float, X::float, Y::float, 0
               FROM (SELECT -2.2 + 0.031 * i, i
                       FROM generate_series(0, 101) AS i) AS xgen(x, ix),
                    (SELECT -1.5 + 0.031 * i, i
                       FROM generate_series(0, 101) AS i) AS ygen(y, iy)
               UNION ALL
               SELECT IX, IY, CX, CY,
                      X * X - Y * Y + CX AS X, Y * X * 2 + CY, I + 1
               FROM Z
               WHERE X * X + Y * Y < 16::float
               AND I < 100
         )
   SELECT array_to_string(array_agg(substring(' .,,,-----++++%%%%@@@@#### ',
          least(greatest(I,1),27), 1)),'')
   FROM (
         SELECT IX, IY, max(I) AS I
         FROM Z
         GROUP BY IY, IX
         ORDER BY IY, IX
        ) AS ZT
   GROUP BY IY
                               Q.                                        ?
   ORDER BY IY;

                                                                               11
WITH                                (                            ):
             ....................................................................................
            .......................................................................................
           .........................................................................................
          ...........................................................................................
        ....................................................,,,,,,,,,.................................
       ................................................,,,,,,,,,,,,,,,,,,.............................
      ..............................................,,,,,,,,,,,,,,,,,,,,,,,,..........................
     ............................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,........................
     ..........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,......................
    .........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,....................
   ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,...................
  .......................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.................
 .......................................,,,,,,,,,,,,,,,,,,,,,,,,--,,,,,,,,,,,,,,,,,,,,................
......................................,,,,,,,,,,,,,,,,,,,,,,,,,,-+--,,,,,,,,,,,,,,,,,,,...............
....................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----,,,,,,,,,,,,,,,,,,,..............
...................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,--- -----,,,,,,,,,,,,,,,,,.............
.................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---++--++,,,,,,,,,,,,,,,,,,............
................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----%++---,,,,,,,,,,,,,,,,,............
..............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%----,,,,,,,,,,,,,,,,,,...........
.............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----- %%+----,,,,,,,,,,,,,,,,,,..........
...........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---%-+%   ----,,,,,,,,,,,,,,,,,,,.........
..........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+ +## %+%---,,,,,,,,,,,,,,,,,,.........
........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----#      # +---,,,,,,,,,,,,,,,,,,........
.......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------%       %-----,,,,,,,,,,,,,,,,,........
.....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---------+         ------,,,,,,,,,,,,,,,,,.......
....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----------+@       +-----------,,,,,,,,,,,,.......
..................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----@-------++       ++-----------,,,,,,,,,,,,......
.................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--+@% ---+ +@%%@     %%+@+@%------+-,,,,,,,,,,,......
................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---- # ++%                % @-----++--,,,,,,,,,,,.....
..............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----+    %                  %%++ %+%@-,,,,,,,,,,,.....
.............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+#                       #%    ++-,,,,,,,,,,,,....
............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,------+                             @---,,,,,,,,,,,,....
..........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------++%                             ---,,,,,,,,,,,,....
.........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+ +                             %+---,,,,,,,,,,,,,...
........,,,,,,,,,,,,,,,,,,,,,--------------------@                                +----,,,,,,,,,,,,...



                                                                                                         A.
.......,,,,,,,,,,,,,,,,,,,,,,- +-----------------+                                 ----,,,,,,,,,,,,...
.......,,,,,,,,,,,,,,,,,,,,,--++------+---------+%                                 +++--,,,,,,,,,,,,..
......,,,,,,,,,,,,,,,,,,,,,,--%+-----++---------                                     #+-,,,,,,,,,,,,..
.....,,,,,,,,,,,,,,,,,,,,,,----#%++--+@ -+-----+%                                     --,,,,,,,,,,,,..
.....,,,,,,,,,,,,,,,,,,,,,,-----+## ++@ + +----%                                    +--,,,,,,,,,,,,,..
....,,,,,,,,,,,,,,,,,,,,,,------+@ @      @@++++#                                   +--,,,,,,,,,,,,,..
....,,,,,,,,,,,,,,,,,,,,,-------%           #++%                                      -,,,,,,,,,,,,,..
...,,,,,,,,,,,,,,,,,,,,,------++%#           %%@                                     %-,,,,,,,,,,,,,,.
...,,,,,,,,,,,,,,,,,,,--------+               %                                     +--,,,,,,,,,,,,,,.
...,,,,,,,,,,,,,,,,,,-----+--++@              #                                      --,,,,,,,,,,,,,,.
..,,,,,,,,,,,,,,,,,-------%+++%                                                    @--,,,,,,,,,,,,,,,.
..,,,,,,,,,,,-------------+ @#@                                                    ---,,,,,,,,,,,,,,,.
..,,,,,,,,,---@--------@-+%                                                       +---,,,,,,,,,,,,,,,.
..,,,,,------- +-++++-+%%%                                                       +----,,,,,,,,,,,,,,,.
..,,,,,,------%--------++%                                                       +----,,,,,,,,,,,,,,,.
..,,,,,,,,,,--+----------++#                                                       ---,,,,,,,,,,,,,,,.
..,,,,,,,,,,,,------------+@@@%                                                    +--,,,,,,,,,,,,,,,.
..,,,,,,,,,,,,,,,,,------- +++%                                                    %--,,,,,,,,,,,,,,,.
...,,,,,,,,,,,,,,,,,,---------+@              @                                      --,,,,,,,,,,,,,,.
...,,,,,,,,,,,,,,,,,,,,------- #              %@                                    +--,,,,,,,,,,,,,,.
...,,,,,,,,,,,,,,,,,,,,,-------++@           %+                                      %-,,,,,,,,,,,,,,.
....,,,,,,,,,,,,,,,,,,,,,-------            %++%                                     %-,,,,,,,,,,,,,..
....,,,,,,,,,,,,,,,,,,,,,,------+# %#    #@ ++++                                    +--,,,,,,,,,,,,,..
.....,,,,,,,,,,,,,,,,,,,,,,-----+ %%++% +@+----+                                    +--,,,,,,,,,,,,,..
.....,,,,,,,,,,,,,,,,,,,,,,,---%+++--+#+--------%                                    #--,,,,,,,,,,,,..
......,,,,,,,,,,,,,,,,,,,,,,--++-----%%---------                                    @#--,,,,,,,,,,,,..
.......,,,,,,,,,,,,,,,,,,,,,---------------------+@                                +-++,,,,,,,,,,,,...
........,,,,,,,,,,,,,,,,,,,,,--------------------+                                 ----,,,,,,,,,,,,...
.........,,,,,,,,,,,,,,,,,,,,----,,,-------------                                #+----,,,,,,,,,,,,...
..........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------+ +                              +---,,,,,,,,,,,,,...
...........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+%#
............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,------+#
.............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+#
..............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+%   %+@
................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----% %@++
                                                                              @
                                                                              +
                                                                                  #---,,,,,,,,,,,,....
                                                                                  @---,,,,,,,,,,,,....
                                                                                   @--,,,,,,,,,,,,....
                                                                            %+-+ +++%-,,,,,,,,,,,.....
                                                                        # % -----++-,,,,,,,,,,,,.....
                                                                                                              WITH   SQL
.................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-- ++ ---+ + +%@     %++++++------%-,,,,,,,,,,,......
...................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---- -------++       +------------,,,,,,,,,,,,......
....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----------+%       +--------,,,,,,,,,,,,,,,.......
......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+#        -----,,,,,,,,,,,,,,,,,,.......
.......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------+       #----,,,,,,,,,,,,,,,,,,........
.........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----+%      %#---,,,,,,,,,,,,,,,,,,,........
..........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+%+%@ %+%%--,,,,,,,,,,,,,,,,,,.........
............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+-+% %----,,,,,,,,,,,,,,,,,,..........
.............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%@+---,,,,,,,,,,,,,,,,,,,..........
...............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%----,,,,,,,,,,,,,,,,,,...........
................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----%+ +--,,,,,,,,,,,,,,,,,............
..................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---++----,,,,,,,,,,,,,,,,,.............
...................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,---@-----,,,,,,,,,,,,,,,,,.............
.....................................,,,,,,,,,,,,,,,,,,,,,,,,,,,-----,,,,,,,,,,,,,,,,,,,..............
 .....................................,,,,,,,,,,,,,,,,,,,,,,,,,,--%,,,,,,,,,,,,,,,,,,,,...............
 .......................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.................
  ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,..................
   ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,...................
    .........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,....................
     ..........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,......................
      ............................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,........................
       .............................................,,,,,,,,,,,,,,,,,,,,,,,,..........................
        ................................................,,,,,,,,,,,,,,,,,.............................
         .....................................................,,,,....................................
          ...........................................................................................
           .........................................................................................
            ......................................................................................
             ....................................................................................
               .................................................................................
                ..............................................................................
                  ...........................................................................
                   ........................................................................




                                                                                                                           12
WITH        (vs. CONNECT BY)
 Oracle CONNECT BY / contrib/tablefunc connectby()
 Oracle : CONNECT BY
  SELECT   level, empno, mgr, ename, job
    FROM   emp
   START   WITH mgr IS NULL
 CONNECT   BY PRIOR empno = mgr

 PostgreSQL ~8.3 : connectby()
 SELECT t.level, e.empno, e.mgr, e.ename, e.job
   FROM emp e, (SELECT * FROM
      connectby('emp', 'empno', 'mgr', 'mgr', 7839, 0, '~')
      AS cb(empno int, mgr int, level int, branch text, pos int)) t
 WHERE e.empno = t.empno;

 PostgreSQL 8.4 : WITH RECURSIVE                  SQL
 WITH RECURSIVE r AS (
     SELECT 1 AS level, * FROM emp WHERE mgr IS NULL
   UNION ALL
     SELECT r.level + 1, emp.* FROM emp, r WHERE r.empno = emp.mgr
 )
 SELECT * FROM r;

                                                                 13
Window           :


   GROUP BY

     (...) OVER (PARTITION BY ...)
     (...) OVER (ORDER BY ...)


  row_number()
  rank()               (               )
  dense_rank()         (                   )
  percent_rank()
                       (%      )
  cume_dist()
  ntile(N)             (1..N       )
                                               self-join
  lag(value, ...)
  lead(value, ...)
  first_value(value)
                                                           14
Window       :


SELECT row_number() OVER (), *
  FROM (SELECT * FROM tbl ORDER BY sortkey) AS t;



    10%
SELECT ... FROM
  (SELECT *, cume_dist() OVER (ORDER BY i) AS rank
   FROM tbl) AS t
WHERE rank <= 0.1;                LIMIT 10%
                                  LIMIT count(*) / 10



SELECT id,    ,     ,
  rank() OVER (PARTITION BY        ORDER BY         )
  FROM tbl;

                                                        15
WITH       + Window                             (1/4)
                      pg_locks

                                            →



                             pg_locks
 =# SELECT locktype, database, relation, page, tuple,
           transactionid, pid, mode, granted
     FROM pg_locks WHERE NOT granted;

 -[ RECORD 1 ]-+--------------   -[ RECORD 2 ]-+--------------
 locktype      | transactionid   locktype      | tuple
 database      |                 database      | 21509
 relation      |                 relation      | 22126
 page          |                 page          | 0
 tuple         |                 tuple         | 1
 transactionid | 9930            transactionid |
 pid           | 3976            pid           | 1992
 mode          | ShareLock       mode          | ExclusiveLock
 granted       | f               granted       | f


                                                                 16
WITH         + Window                            (2/4)
                         locktag()

CREATE FUNCTION locktag(pg_catalog.pg_locks) RETURNS text AS
$$
SELECT $1.locktype || ' ' ||
  CASE $1.locktype
    WHEN 'relation'      THEN $1.database || ' ' || $1.relation
    WHEN 'extend'        THEN $1.database || ' ' || $1.relation
    WHEN 'page'          THEN $1.database || ' ' || $1.relation || ' ' || $1.page
    WHEN 'tuple'         THEN $1.database || ' ' || $1.relation || ' ' || $1.page
                              || ' ' || $1.tuple
    WHEN 'transactionid' THEN $1.transactionid::text
    WHEN 'virtualxid'    THEN $1.virtualxid
    -- FIXME: for object, userlock and advisory
  END
$$
LANGUAGE sql IMMUTABLE STRICT;




                                                                               17
WITH      + Window                     (3/4)
          pg_lock_chain
                  row_number()
CREATE VIEW pg_lock_chain AS
  WITH RECURSIVE r AS (
      SELECT *, locktag(pg_locks),
             row_number() OVER () AS chain,
             1 AS level
       FROM pg_locks WHERE NOT granted
    UNION ALL
      SELECT s.*, locktag(s), r.chain, r.level + 1
        FROM r, pg_locks s
       WHERE (locktag(s) = r.locktag AND s.granted AND
              NOT r.granted AND s.pid <> r.pid)
          OR (s.pid = r.pid AND NOT s.granted AND r.granted)
     )
   SELECT * FROM r;                           locktag



                                                           18
WITH       + Window                            (4/4)
              pg_lock_chain
=# SELECT chain, level, locktag, pid, mode, granted
     FROM pg_lock_chain
    ORDER BY chain, level;

 chain | level |        locktag        | pid |      mode      | granted
-------+-------+-----------------------+------+---------------+---------
     1 |     1 | transactionid 9930    | 3976 | ShareLock     | f
     1 |     2 | transactionid 9930    | 3912 | ExclusiveLock | t
     2 |     1 | tuple 21509 22126 0 1 | 1992 | ExclusiveLock | f
     2 |     2 | tuple 21509 22126 0 1 | 3976 | ExclusiveLock | t
     2 |     3 | transactionid 9930    | 3976 | ShareLock     | f
     2 |     4 | transactionid 9930    | 3912 | ExclusiveLock | t
(6 rows)

        chain 1              chain 2
                  pid=3976             pid=1992
                                                          pid=3912
                  pid=3912             pid=3976

                                       pid=3912

                                                                     19
/
PostgreSQL 8.2


    CREATE CAST (integer AS varchar)
                  WITH INOUT AS IMPLICIT;
 DBMS
                  AS
    SELECT   [AS]      FROM   [AS]
  DISTINCT GROUP BY
                 or
  PL/pgSQL CASE

                                            20
=# SELECT *, to_tsvector('japanese', t) FROM tbl;
 id |     t        | to_tsvector
----+------------+----------------
  1 |              | '            ':1
  2 |              | '          ':1
(2 rows)

=# SELECT * FROM tbl WHERE to_tsvector('japanese', t) @@ to_tsquery('japanese', '   ');
 id |    t
----+----------
  2 |
(1 row)

=# SELECT * FROM tbl WHERE to_tsvector('japanese', t) @@ to_tsquery('japanese', '   :*');
 id |     t
----+------------
  1 |                                                                 :*
  2 |
(2 rows)

   N-gram                                                           ( :Ludia/Senna)


                                                                                          21
PostgreSQL 8.4 :


             /
                          WITH
                        Window
                             /




      SQL                 pgtune
       Visibility Map    pg_reorg




                                    22
SQL              :


   pg_stat_statements              : SQL
   pg_stat_user_functions          :

               SQL

                               pgFouine   PHP
  PostgreSQL               (               )
               SQL

   ◎                       ☺                    (HTML   )
   ×
   ×                           /
       DB            30%
                                                            23
SQL                          :
                   postgresql.conf
     shared_preload_libraries = 'pg_stat_statements'
                                                                 prepared
                    pgbench                                   (Prepared Statement)
     $ pgbench -c10 -t300 -M prepared
     query mode: prepared
     number of transactions actually processed: 3000/3000

    SQL
=# SELECT query, calls, total_time, rows FROM pg_stat_statements ORDER BY total_time DESC LIMIT 3;
                            query                             | calls |    total_time     | rows
--------------------------------------------------------------+-------+-------------------+------
 UPDATE branches SET bbalance = bbalance + $1 WHERE bid = $2; | 3000 | 35.9654100452473 | 3000
 UPDATE tellers SET tbalance = tbalance + $1 WHERE tid = $2; | 3000 | 34.7969816235719 | 3000
 UPDATE accounts SET abalance = abalance + $1 WHERE aid = $2; | 3000 | 0.660384690059746 | 3000
(3 rows)

                             SQL


        ◎
        ◎                                             3%
        × HTML
                                                                                                24
Visibility Map : VACUUM
 PostgreSQL VACUUM                     ?


                ID                      ID
               001   A       10        001   A       10
               002   B        8
UPDATE         003   C       20        003   C       20       VACUUM
               002   B        7        002   B        7



            8.4      VACUUM
    Visibility Map
                                  VACUUM                  +
    Free Space Map

            8.3      HOT                         ?
    HOT                  =
    Visibility Map       =
                                                                       25
VACUUM
 Visibility Map (VACUUM                  )
   8.4               VACUUM
                                              VACUUM
 Free Space Map (VACUUM                       )

                           8.4
                                        DB
                                 Visibility Map       Free Space Map
    :    (<   ID>[.N])           (<      ID>_vm)       (<          ID>_fsm)

                                      1bit/              0%
                                                        50% 1byte/
                                                        25%

                                                            Visibility Map
                                             VACUUM                 Free Space Map



                         (shared_buffers)                                     I/O
                                                                                     26
SQL
                                                  SQL

 id   value                       1 (2392)                       2 (pid=1956)
                   BEGIN;                           BEGIN;
 1      A
                   UPDATE tbl SET value = 'AA'
 2      B           WHERE id = 1;                   UPDATE tbl SET value = 'BB'
                                                     WHERE id = 2;
                   UPDATE tbl SET value = 'BBB'
                    WHERE id = 2;
                                                    UPDATE tbl SET value = 'AAA'
                                                     WHERE id = 1;

ERROR: deadlock detected
DETAIL:
      Process 1956 waits for ShareLock on transaction 8214; blocked by process 2392.
      Process 2392 waits for ShareLock on transaction 8215; blocked by process 1956.
      Process 1956: UPDATE tbl SET value = 'AAA' WHERE id = 1;
      Process 2392: UPDATE tbl SET value = 'BBB' WHERE id = 2;



                                                                                       27
PostgreSQL 8.4 :


             /
                          WITH
                        Window
                             /




      SQL                 pgtune
       Visibility Map    pg_reorg




                                    28
pgtune :



                (DW, OLTP, Web, Mixed, Desktop)
                  (        )                                        postgresql.conf
                                                                     postgresql.conf
               =Mixed,       =3GB
                                                  pgtune                 in
      max_connections                     100            80
      shared_buffers                    32MB       480MB               pgtune
      wal_buffers                        64kB        8MB                out
      work_mem                           1MB        12MB
      maintenance_work_mem              16MB       120MB
                                                                    postgresql.conf
                                                                     postgresql.conf
      checkpoint_segments                    3           16
WAL
      checkpoint_completion_target         0.5        0.9
      effective_cache_size            128MB 1408MB
      constraint_exclusion           partition      on
      default_statistics_target           100            50

                                                 http://pgfoundry.org/projects/pgtune/   29
pg_reorg :

                                    /
                                         DB
         VACUUM




         INSERT
         UPDATE
         DELETE                                                (                )

                               (          2GB)
■ clusterdb   274
■ pg_reorg    17


                                                           PostgreSQL: 8.2, 8.3, 8.4
                    0   100   200       300                  OS: Linux, Windows


                                    http://reorg.projects.postgresql.org/index-ja.html   30
pgbench :
 pgbench
                  TPC-B
            SQL



 pgbench
                   (-T)
                            (-t)



                     (-M)
      simple, extended, prepared
                                   31
PostgreSQL 8.4 (                         )
        2009 7
          /
                   ,                     ,

    WITH , Window       ,            /

    SQL          , Visibility Map,

    pgtune, pg_reorg




                                             32
TRUNCATE TRIGGER
SQL/MED


pg_terminate_backend()
contrib/auto_explain


autovacuum               TOAST         VACUUM



pgAdmin III                      pgScript

                                                33
ToDo (                                   )
  http://wiki.postgresql.org/wiki/Todo
PostgreSQL 8.4 Development Plan (              )
  http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Dev
  elopment_Plan
Waiting for 8.4 (                blog)
  http://www.depesz.com/




                                                   34

Mais conteúdo relacionado

Semelhante a PostgreSQL 8.4 Update

Intro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialIntro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialRobert Treat
 
โครงงานคอมพิวเตอร์ ฉบับจริง
โครงงานคอมพิวเตอร์ ฉบับจริงโครงงานคอมพิวเตอร์ ฉบับจริง
โครงงานคอมพิวเตอร์ ฉบับจริงThepond Incledible
 
A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0Robert Treat
 
Intro to Postgres 8.4 Tutorial
Intro to Postgres 8.4 TutorialIntro to Postgres 8.4 Tutorial
Intro to Postgres 8.4 TutorialRobert Treat
 
Objectclub2009summer Youth Session Takkanm
Objectclub2009summer Youth Session TakkanmObjectclub2009summer Youth Session Takkanm
Objectclub2009summer Youth Session Takkanmmitsutaka mimura
 
10x improvement-mysql-100419105218-phpapp02
10x improvement-mysql-100419105218-phpapp0210x improvement-mysql-100419105218-phpapp02
10x improvement-mysql-100419105218-phpapp02promethius
 
10x Performance Improvements
10x Performance Improvements10x Performance Improvements
10x Performance ImprovementsRonald Bradford
 
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...JSFestUA
 
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...Lars Aksel Opsahl
 
経営者から見たクラウド 2011年6月
経営者から見たクラウド 2011年6月経営者から見たクラウド 2011年6月
経営者から見たクラウド 2011年6月Serverworks Co.,Ltd.
 
Ecos基础应用介绍
Ecos基础应用介绍Ecos基础应用介绍
Ecos基础应用介绍wanglei999
 
DevLOVE Beautiful Development - 第一幕 陽の巻
DevLOVE Beautiful Development - 第一幕 陽の巻DevLOVE Beautiful Development - 第一幕 陽の巻
DevLOVE Beautiful Development - 第一幕 陽の巻都元ダイスケ Miyamoto
 
Getting ready for the new MySQL
Getting ready for the new MySQLGetting ready for the new MySQL
Getting ready for the new MySQLGiuseppe Maxia
 
Migrating from PostgreSQL to MySQL at Cocolog
Migrating from PostgreSQL to MySQL at CocologMigrating from PostgreSQL to MySQL at Cocolog
Migrating from PostgreSQL to MySQL at CocologNaoto Yokoyama
 
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Continuent
 
Postgres Plus Advanced Server 9.2新機能ご紹介
Postgres Plus Advanced Server 9.2新機能ご紹介Postgres Plus Advanced Server 9.2新機能ご紹介
Postgres Plus Advanced Server 9.2新機能ご紹介Yuji Fujita
 
IPW2008 - my.opera.com scalability
IPW2008 - my.opera.com scalabilityIPW2008 - my.opera.com scalability
IPW2008 - my.opera.com scalabilityCosimo Streppone
 

Semelhante a PostgreSQL 8.4 Update (20)

Intro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialIntro to Postgres 9 Tutorial
Intro to Postgres 9 Tutorial
 
Git tutorial
Git tutorialGit tutorial
Git tutorial
 
โครงงานคอมพิวเตอร์ ฉบับจริง
โครงงานคอมพิวเตอร์ ฉบับจริงโครงงานคอมพิวเตอร์ ฉบับจริง
โครงงานคอมพิวเตอร์ ฉบับจริง
 
A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0
 
Intro to Postgres 8.4 Tutorial
Intro to Postgres 8.4 TutorialIntro to Postgres 8.4 Tutorial
Intro to Postgres 8.4 Tutorial
 
MateriApps LIVE!の設定
MateriApps LIVE!の設定MateriApps LIVE!の設定
MateriApps LIVE!の設定
 
SCALE 10x Build a Cloud Day
SCALE 10x Build a Cloud DaySCALE 10x Build a Cloud Day
SCALE 10x Build a Cloud Day
 
Objectclub2009summer Youth Session Takkanm
Objectclub2009summer Youth Session TakkanmObjectclub2009summer Youth Session Takkanm
Objectclub2009summer Youth Session Takkanm
 
10x improvement-mysql-100419105218-phpapp02
10x improvement-mysql-100419105218-phpapp0210x improvement-mysql-100419105218-phpapp02
10x improvement-mysql-100419105218-phpapp02
 
10x Performance Improvements
10x Performance Improvements10x Performance Improvements
10x Performance Improvements
 
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...
JS Fest 2019. Олег Докука и Даниил Дробот. RSocket - future Reactive Applicat...
 
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...
Postgis_Topology_to_secure_data_integrity,_simple_API_and_clean_up_messy_simp...
 
経営者から見たクラウド 2011年6月
経営者から見たクラウド 2011年6月経営者から見たクラウド 2011年6月
経営者から見たクラウド 2011年6月
 
Ecos基础应用介绍
Ecos基础应用介绍Ecos基础应用介绍
Ecos基础应用介绍
 
DevLOVE Beautiful Development - 第一幕 陽の巻
DevLOVE Beautiful Development - 第一幕 陽の巻DevLOVE Beautiful Development - 第一幕 陽の巻
DevLOVE Beautiful Development - 第一幕 陽の巻
 
Getting ready for the new MySQL
Getting ready for the new MySQLGetting ready for the new MySQL
Getting ready for the new MySQL
 
Migrating from PostgreSQL to MySQL at Cocolog
Migrating from PostgreSQL to MySQL at CocologMigrating from PostgreSQL to MySQL at Cocolog
Migrating from PostgreSQL to MySQL at Cocolog
 
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
 
Postgres Plus Advanced Server 9.2新機能ご紹介
Postgres Plus Advanced Server 9.2新機能ご紹介Postgres Plus Advanced Server 9.2新機能ご紹介
Postgres Plus Advanced Server 9.2新機能ご紹介
 
IPW2008 - my.opera.com scalability
IPW2008 - my.opera.com scalabilityIPW2008 - my.opera.com scalability
IPW2008 - my.opera.com scalability
 

Último

08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 

Último (20)

08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 

PostgreSQL 8.4 Update

  • 1. LOCAL DEVELOPER DAY '09 /Winter ( @ , 2009 ) http://www.local.gr.jp/?php%2Fhistory%2F%C2%E803%B2%F3%28LDD%2709%2FWinter%29 PostgreSQL 8.4 Update PostgreSQL / NTT OSS 2009.2.14
  • 2. PostgreSQL 8.4 8.0 8.3 8.4 2009 7 Ingress Ingress PostgreSQL PostgreSQL 8.1 6.0 6.0 •• 1977 7.4 7.4 •2 (2PC) 8.3 1986 1996 •2 (2PC) 2000 •• •HOT: •HOT: 2003 2004 • • POSTGRES POSTGRES 2005 •VACUUM •VACUUM // 2006 7.3 7.3 2007 2008 8.0 •• 2009 •• (PITR) (PITR) •• 8.2 •Windows •CPU •CPU •Windows • • • • •GIN: •GIN: 8.4 2
  • 3. 8.4 / HOT XML autovacuum 8.3 2008 WITH SQL pgtune 8.4 Window Visibility Map pg_reorg 2009 / 8.5 2010 ? 3
  • 4. PostgreSQL 8.4 : / / / WITH Window / SQL pgtune Visibility Map pg_reorg 4
  • 5. : / CHECK 8.2 • SELECT, UPDATE, DELETE • 1 2 3 4 4× ( ) 8.4 constraint_exclusion = partition CHECK EXECUTE USING in PL/pgSQL INSERT 5
  • 6. : (EXECUTE USING) / INSERT : tbl CREATE TRIGGER BEFORE INSERT ( ) CHECK CHECK('2009-01-01' <= insert_time 1 2 3 4 AND insert_time < '2009-02-01') : tbl_[ ][ ] CREATE FUNCTION insert_trigger() RETURNS TRIGGER AS $$ DECLARE part text; (8.3 BEGIN IF ) part := 'tbl_' || to_char(new.insert_time, 'YYYYMM'); EXECUTE 'INSERT INTO ' || part || ' VALUES(($1).*)' USING new; RETURN NULL; END; $$ EXECUTE USING LANGUAGE plpgsql; SQL new 6
  • 7. : / effective_io_concurrency I/O RAID I/O Bitmap Heap Scan ◎1 SQL OLTP × OLAP posix_fadvise(WILLNEED) POSIX ( Linux) / Windows 7
  • 8. / pg_restore -m, --multi-thread = number-of-threads PostgreSQL CPU PostgreSQL pg_dump --format=custom D pg_restore R R R R ( ) ( ) 8
  • 9. PostgreSQL 8.4 : / WITH Window / SQL pgtune Visibility Map pg_reorg 9
  • 10. WITH ( ) WITH, WITH RECURSIVE ID =# TABLE tree; id | parent 1 ----+-------- 1 | 2 | 1 2 3 self-join 3 | 1 4 | 3 4 ... =# WITH RECURSIVE r AS ( SELECT * FROM tree WHERE id = 1 UNION ALL SELECT tree.* FROM tree, r WHERE tree.parent = r.id ) SELECT * FROM r ORDER BY id; id | parent ----+-------- 1 | 2 | 1 3 | 1 4 | 3 (4 rows) 10
  • 11. WITH ( ): ML (!?) WITH RECURSIVE Z(IX, IY, CX, CY, X, Y, I) AS ( SELECT IX, IY, X::float, Y::float, X::float, Y::float, 0 FROM (SELECT -2.2 + 0.031 * i, i FROM generate_series(0, 101) AS i) AS xgen(x, ix), (SELECT -1.5 + 0.031 * i, i FROM generate_series(0, 101) AS i) AS ygen(y, iy) UNION ALL SELECT IX, IY, CX, CY, X * X - Y * Y + CX AS X, Y * X * 2 + CY, I + 1 FROM Z WHERE X * X + Y * Y < 16::float AND I < 100 ) SELECT array_to_string(array_agg(substring(' .,,,-----++++%%%%@@@@#### ', least(greatest(I,1),27), 1)),'') FROM ( SELECT IX, IY, max(I) AS I FROM Z GROUP BY IY, IX ORDER BY IY, IX ) AS ZT GROUP BY IY Q. ? ORDER BY IY; 11
  • 12. WITH ( ): .................................................................................... ....................................................................................... ......................................................................................... ........................................................................................... ....................................................,,,,,,,,,................................. ................................................,,,,,,,,,,,,,,,,,,............................. ..............................................,,,,,,,,,,,,,,,,,,,,,,,,.......................... ............................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,........................ ..........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,...................... .........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.................... ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,................... .......................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,................. .......................................,,,,,,,,,,,,,,,,,,,,,,,,--,,,,,,,,,,,,,,,,,,,,................ ......................................,,,,,,,,,,,,,,,,,,,,,,,,,,-+--,,,,,,,,,,,,,,,,,,,............... ....................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----,,,,,,,,,,,,,,,,,,,.............. ...................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,--- -----,,,,,,,,,,,,,,,,,............. .................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---++--++,,,,,,,,,,,,,,,,,,............ ................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----%++---,,,,,,,,,,,,,,,,,............ ..............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%----,,,,,,,,,,,,,,,,,,........... .............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----- %%+----,,,,,,,,,,,,,,,,,,.......... ...........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---%-+% ----,,,,,,,,,,,,,,,,,,,......... ..........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+ +## %+%---,,,,,,,,,,,,,,,,,,......... ........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----# # +---,,,,,,,,,,,,,,,,,,........ .......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------% %-----,,,,,,,,,,,,,,,,,........ .....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---------+ ------,,,,,,,,,,,,,,,,,....... ....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----------+@ +-----------,,,,,,,,,,,,....... ..................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----@-------++ ++-----------,,,,,,,,,,,,...... .................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--+@% ---+ +@%%@ %%+@+@%------+-,,,,,,,,,,,...... ................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---- # ++% % @-----++--,,,,,,,,,,,..... ..............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----+ % %%++ %+%@-,,,,,,,,,,,..... .............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+# #% ++-,,,,,,,,,,,,.... ............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,------+ @---,,,,,,,,,,,,.... ..........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------++% ---,,,,,,,,,,,,.... .........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+ + %+---,,,,,,,,,,,,,... ........,,,,,,,,,,,,,,,,,,,,,--------------------@ +----,,,,,,,,,,,,... A. .......,,,,,,,,,,,,,,,,,,,,,,- +-----------------+ ----,,,,,,,,,,,,... .......,,,,,,,,,,,,,,,,,,,,,--++------+---------+% +++--,,,,,,,,,,,,.. ......,,,,,,,,,,,,,,,,,,,,,,--%+-----++--------- #+-,,,,,,,,,,,,.. .....,,,,,,,,,,,,,,,,,,,,,,----#%++--+@ -+-----+% --,,,,,,,,,,,,.. .....,,,,,,,,,,,,,,,,,,,,,,-----+## ++@ + +----% +--,,,,,,,,,,,,,.. ....,,,,,,,,,,,,,,,,,,,,,,------+@ @ @@++++# +--,,,,,,,,,,,,,.. ....,,,,,,,,,,,,,,,,,,,,,-------% #++% -,,,,,,,,,,,,,.. ...,,,,,,,,,,,,,,,,,,,,,------++%# %%@ %-,,,,,,,,,,,,,,. ...,,,,,,,,,,,,,,,,,,,--------+ % +--,,,,,,,,,,,,,,. ...,,,,,,,,,,,,,,,,,,-----+--++@ # --,,,,,,,,,,,,,,. ..,,,,,,,,,,,,,,,,,-------%+++% @--,,,,,,,,,,,,,,,. ..,,,,,,,,,,,-------------+ @#@ ---,,,,,,,,,,,,,,,. ..,,,,,,,,,---@--------@-+% +---,,,,,,,,,,,,,,,. ..,,,,,------- +-++++-+%%% +----,,,,,,,,,,,,,,,. ..,,,,,,------%--------++% +----,,,,,,,,,,,,,,,. ..,,,,,,,,,,--+----------++# ---,,,,,,,,,,,,,,,. ..,,,,,,,,,,,,------------+@@@% +--,,,,,,,,,,,,,,,. ..,,,,,,,,,,,,,,,,,------- +++% %--,,,,,,,,,,,,,,,. ...,,,,,,,,,,,,,,,,,,---------+@ @ --,,,,,,,,,,,,,,. ...,,,,,,,,,,,,,,,,,,,,------- # %@ +--,,,,,,,,,,,,,,. ...,,,,,,,,,,,,,,,,,,,,,-------++@ %+ %-,,,,,,,,,,,,,,. ....,,,,,,,,,,,,,,,,,,,,,------- %++% %-,,,,,,,,,,,,,.. ....,,,,,,,,,,,,,,,,,,,,,,------+# %# #@ ++++ +--,,,,,,,,,,,,,.. .....,,,,,,,,,,,,,,,,,,,,,,-----+ %%++% +@+----+ +--,,,,,,,,,,,,,.. .....,,,,,,,,,,,,,,,,,,,,,,,---%+++--+#+--------% #--,,,,,,,,,,,,.. ......,,,,,,,,,,,,,,,,,,,,,,--++-----%%--------- @#--,,,,,,,,,,,,.. .......,,,,,,,,,,,,,,,,,,,,,---------------------+@ +-++,,,,,,,,,,,,... ........,,,,,,,,,,,,,,,,,,,,,--------------------+ ----,,,,,,,,,,,,... .........,,,,,,,,,,,,,,,,,,,,----,,,------------- #+----,,,,,,,,,,,,... ..........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------+ + +---,,,,,,,,,,,,,... ...........,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+%# ............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,------+# .............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+# ..............,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+% %+@ ................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----% %@++ @ + #---,,,,,,,,,,,,.... @---,,,,,,,,,,,,.... @--,,,,,,,,,,,,.... %+-+ +++%-,,,,,,,,,,,..... # % -----++-,,,,,,,,,,,,..... WITH SQL .................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-- ++ ---+ + +%@ %++++++------%-,,,,,,,,,,,...... ...................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---- -------++ +------------,,,,,,,,,,,,...... ....................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----------+% +--------,,,,,,,,,,,,,,,....... ......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,--------+# -----,,,,,,,,,,,,,,,,,,....... .......................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-------+ #----,,,,,,,,,,,,,,,,,,........ .........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,----+% %#---,,,,,,,,,,,,,,,,,,,........ ..........................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+%+%@ %+%%--,,,,,,,,,,,,,,,,,,......... ............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---+-+% %----,,,,,,,,,,,,,,,,,,.......... .............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%@+---,,,,,,,,,,,,,,,,,,,.......... ...............................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----+%----,,,,,,,,,,,,,,,,,,........... ................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-----%+ +--,,,,,,,,,,,,,,,,,............ ..................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,---++----,,,,,,,,,,,,,,,,,............. ...................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,---@-----,,,,,,,,,,,,,,,,,............. .....................................,,,,,,,,,,,,,,,,,,,,,,,,,,,-----,,,,,,,,,,,,,,,,,,,.............. .....................................,,,,,,,,,,,,,,,,,,,,,,,,,,--%,,,,,,,,,,,,,,,,,,,,............... .......................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,................. ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.................. ........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,................... .........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,.................... ..........................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,...................... ............................................,,,,,,,,,,,,,,,,,,,,,,,,,,,,........................ .............................................,,,,,,,,,,,,,,,,,,,,,,,,.......................... ................................................,,,,,,,,,,,,,,,,,............................. .....................................................,,,,.................................... ........................................................................................... ......................................................................................... ...................................................................................... .................................................................................... ................................................................................. .............................................................................. ........................................................................... ........................................................................ 12
  • 13. WITH (vs. CONNECT BY) Oracle CONNECT BY / contrib/tablefunc connectby() Oracle : CONNECT BY SELECT level, empno, mgr, ename, job FROM emp START WITH mgr IS NULL CONNECT BY PRIOR empno = mgr PostgreSQL ~8.3 : connectby() SELECT t.level, e.empno, e.mgr, e.ename, e.job FROM emp e, (SELECT * FROM connectby('emp', 'empno', 'mgr', 'mgr', 7839, 0, '~') AS cb(empno int, mgr int, level int, branch text, pos int)) t WHERE e.empno = t.empno; PostgreSQL 8.4 : WITH RECURSIVE SQL WITH RECURSIVE r AS ( SELECT 1 AS level, * FROM emp WHERE mgr IS NULL UNION ALL SELECT r.level + 1, emp.* FROM emp, r WHERE r.empno = emp.mgr ) SELECT * FROM r; 13
  • 14. Window : GROUP BY (...) OVER (PARTITION BY ...) (...) OVER (ORDER BY ...) row_number() rank() ( ) dense_rank() ( ) percent_rank() (% ) cume_dist() ntile(N) (1..N ) self-join lag(value, ...) lead(value, ...) first_value(value) 14
  • 15. Window : SELECT row_number() OVER (), * FROM (SELECT * FROM tbl ORDER BY sortkey) AS t; 10% SELECT ... FROM (SELECT *, cume_dist() OVER (ORDER BY i) AS rank FROM tbl) AS t WHERE rank <= 0.1; LIMIT 10% LIMIT count(*) / 10 SELECT id, , , rank() OVER (PARTITION BY ORDER BY ) FROM tbl; 15
  • 16. WITH + Window (1/4) pg_locks → pg_locks =# SELECT locktype, database, relation, page, tuple, transactionid, pid, mode, granted FROM pg_locks WHERE NOT granted; -[ RECORD 1 ]-+-------------- -[ RECORD 2 ]-+-------------- locktype | transactionid locktype | tuple database | database | 21509 relation | relation | 22126 page | page | 0 tuple | tuple | 1 transactionid | 9930 transactionid | pid | 3976 pid | 1992 mode | ShareLock mode | ExclusiveLock granted | f granted | f 16
  • 17. WITH + Window (2/4) locktag() CREATE FUNCTION locktag(pg_catalog.pg_locks) RETURNS text AS $$ SELECT $1.locktype || ' ' || CASE $1.locktype WHEN 'relation' THEN $1.database || ' ' || $1.relation WHEN 'extend' THEN $1.database || ' ' || $1.relation WHEN 'page' THEN $1.database || ' ' || $1.relation || ' ' || $1.page WHEN 'tuple' THEN $1.database || ' ' || $1.relation || ' ' || $1.page || ' ' || $1.tuple WHEN 'transactionid' THEN $1.transactionid::text WHEN 'virtualxid' THEN $1.virtualxid -- FIXME: for object, userlock and advisory END $$ LANGUAGE sql IMMUTABLE STRICT; 17
  • 18. WITH + Window (3/4) pg_lock_chain row_number() CREATE VIEW pg_lock_chain AS WITH RECURSIVE r AS ( SELECT *, locktag(pg_locks), row_number() OVER () AS chain, 1 AS level FROM pg_locks WHERE NOT granted UNION ALL SELECT s.*, locktag(s), r.chain, r.level + 1 FROM r, pg_locks s WHERE (locktag(s) = r.locktag AND s.granted AND NOT r.granted AND s.pid <> r.pid) OR (s.pid = r.pid AND NOT s.granted AND r.granted) ) SELECT * FROM r; locktag 18
  • 19. WITH + Window (4/4) pg_lock_chain =# SELECT chain, level, locktag, pid, mode, granted FROM pg_lock_chain ORDER BY chain, level; chain | level | locktag | pid | mode | granted -------+-------+-----------------------+------+---------------+--------- 1 | 1 | transactionid 9930 | 3976 | ShareLock | f 1 | 2 | transactionid 9930 | 3912 | ExclusiveLock | t 2 | 1 | tuple 21509 22126 0 1 | 1992 | ExclusiveLock | f 2 | 2 | tuple 21509 22126 0 1 | 3976 | ExclusiveLock | t 2 | 3 | transactionid 9930 | 3976 | ShareLock | f 2 | 4 | transactionid 9930 | 3912 | ExclusiveLock | t (6 rows) chain 1 chain 2 pid=3976 pid=1992 pid=3912 pid=3912 pid=3976 pid=3912 19
  • 20. / PostgreSQL 8.2 CREATE CAST (integer AS varchar) WITH INOUT AS IMPLICIT; DBMS AS SELECT [AS] FROM [AS] DISTINCT GROUP BY or PL/pgSQL CASE 20
  • 21. =# SELECT *, to_tsvector('japanese', t) FROM tbl; id | t | to_tsvector ----+------------+---------------- 1 | | ' ':1 2 | | ' ':1 (2 rows) =# SELECT * FROM tbl WHERE to_tsvector('japanese', t) @@ to_tsquery('japanese', ' '); id | t ----+---------- 2 | (1 row) =# SELECT * FROM tbl WHERE to_tsvector('japanese', t) @@ to_tsquery('japanese', ' :*'); id | t ----+------------ 1 | :* 2 | (2 rows) N-gram ( :Ludia/Senna) 21
  • 22. PostgreSQL 8.4 : / WITH Window / SQL pgtune Visibility Map pg_reorg 22
  • 23. SQL : pg_stat_statements : SQL pg_stat_user_functions : SQL pgFouine PHP PostgreSQL ( ) SQL ◎ ☺ (HTML ) × × / DB 30% 23
  • 24. SQL : postgresql.conf shared_preload_libraries = 'pg_stat_statements' prepared pgbench (Prepared Statement) $ pgbench -c10 -t300 -M prepared query mode: prepared number of transactions actually processed: 3000/3000 SQL =# SELECT query, calls, total_time, rows FROM pg_stat_statements ORDER BY total_time DESC LIMIT 3; query | calls | total_time | rows --------------------------------------------------------------+-------+-------------------+------ UPDATE branches SET bbalance = bbalance + $1 WHERE bid = $2; | 3000 | 35.9654100452473 | 3000 UPDATE tellers SET tbalance = tbalance + $1 WHERE tid = $2; | 3000 | 34.7969816235719 | 3000 UPDATE accounts SET abalance = abalance + $1 WHERE aid = $2; | 3000 | 0.660384690059746 | 3000 (3 rows) SQL ◎ ◎ 3% × HTML 24
  • 25. Visibility Map : VACUUM PostgreSQL VACUUM ? ID ID 001 A 10 001 A 10 002 B 8 UPDATE 003 C 20 003 C 20 VACUUM 002 B 7 002 B 7 8.4 VACUUM Visibility Map VACUUM + Free Space Map 8.3 HOT ? HOT = Visibility Map = 25
  • 26. VACUUM Visibility Map (VACUUM ) 8.4 VACUUM VACUUM Free Space Map (VACUUM ) 8.4 DB Visibility Map Free Space Map : (< ID>[.N]) (< ID>_vm) (< ID>_fsm) 1bit/ 0% 50% 1byte/ 25% Visibility Map VACUUM Free Space Map (shared_buffers) I/O 26
  • 27. SQL SQL id value 1 (2392) 2 (pid=1956) BEGIN; BEGIN; 1 A UPDATE tbl SET value = 'AA' 2 B WHERE id = 1; UPDATE tbl SET value = 'BB' WHERE id = 2; UPDATE tbl SET value = 'BBB' WHERE id = 2; UPDATE tbl SET value = 'AAA' WHERE id = 1; ERROR: deadlock detected DETAIL: Process 1956 waits for ShareLock on transaction 8214; blocked by process 2392. Process 2392 waits for ShareLock on transaction 8215; blocked by process 1956. Process 1956: UPDATE tbl SET value = 'AAA' WHERE id = 1; Process 2392: UPDATE tbl SET value = 'BBB' WHERE id = 2; 27
  • 28. PostgreSQL 8.4 : / WITH Window / SQL pgtune Visibility Map pg_reorg 28
  • 29. pgtune : (DW, OLTP, Web, Mixed, Desktop) ( ) postgresql.conf postgresql.conf =Mixed, =3GB pgtune in max_connections 100 80 shared_buffers 32MB 480MB pgtune wal_buffers 64kB 8MB out work_mem 1MB 12MB maintenance_work_mem 16MB 120MB postgresql.conf postgresql.conf checkpoint_segments 3 16 WAL checkpoint_completion_target 0.5 0.9 effective_cache_size 128MB 1408MB constraint_exclusion partition on default_statistics_target 100 50 http://pgfoundry.org/projects/pgtune/ 29
  • 30. pg_reorg : / DB VACUUM INSERT UPDATE DELETE ( ) ( 2GB) ■ clusterdb 274 ■ pg_reorg 17 PostgreSQL: 8.2, 8.3, 8.4 0 100 200 300 OS: Linux, Windows http://reorg.projects.postgresql.org/index-ja.html 30
  • 31. pgbench : pgbench TPC-B SQL pgbench (-T) (-t) (-M) simple, extended, prepared 31
  • 32. PostgreSQL 8.4 ( ) 2009 7 / , , WITH , Window , / SQL , Visibility Map, pgtune, pg_reorg 32
  • 34. ToDo ( ) http://wiki.postgresql.org/wiki/Todo PostgreSQL 8.4 Development Plan ( ) http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Dev elopment_Plan Waiting for 8.4 ( blog) http://www.depesz.com/ 34