Detailed presentation devoted to new features of Firebird 2.1 by Vladislav Khorsun, core Firebird Developer. Main features are covered, including tips and tricks and often usage scenarios.
11.
DATABASE TRIGGERS
Example of ON CONNECT trigger
isql temp.fdb -user SYSDBA -pass masterkey
Database: temp .fdb, User: SYSDBA
SQL> SET TERM ^ ;
SQL> CREATE EXCEPTION EX_CONNECT 'Forbidden !' ^
SQL> CREATE OR ALTER TRIGGER TRG_CONN ON CONNECT
CON> AS
CON> BEGIN
CON> IF (<bad user>)
CON> THEN EXCEPTION EX_CONNECT USER || ' not allowed !';
CON> END ^
SQL> EXIT ^
isql temp.fdb -user BAD_USER -pass ...
Statement failed, SQLCODE = -836
exception 217
-EX_CONNECT
- BAD_USER not allowed !
-At trigger 'TRG_CONN' line: 5, col: 3
Use CONNECT or CREATE DATABASE to specify a database
SQL> EXIT;
12.
13.
14.
15.
GLOBAL TEMPORARY TABLES
Referential integrity rules
on commit delete rows
on commit preserve rows
persistent
on commit delete rows
on commit preserve rows
persistent
master
detail
16.
GLOBAL TEMPORARY TABLES
database
CREATE GLOBAL TEMPORARY TABLE T1
attachment 1
attachment 2
pages not allocated
pages not allocated
pages allocated ( temporary file 1, FW = OFF )
pages freed ( temporary file deleted )
INSERT INTO T1
SELECT FROM T1
pages allocated
disconnect
pages freed ( temporary file deleted )
INSERT INTO T1
SELECT FROM T1
pages allocated ( temporary file 2, FW = OFF )
disconnect
pages not allocated
Common metadata
Private data
17.
18.
19.
20.
COMMON TABLE EXPRESSIONS
Example of simple ( non recursive ) table expressions
WITH
DEPT_YEAR_BUDGET AS (
SELECT FISCAL_YEAR, DEPT_NO, SUM(PROJECTED_BUDGET) AS BUDGET
FROM PROJ_DEPT_BUDGET
GROUP BY FISCAL_YEAR, DEPT_NO
)
SELECT D.DEPT_NO, D.DEPARTMENT,
B_1993.BUDGET AS B_1993, B_1994.BUDGET AS B_1994,
B_1995.BUDGET AS B_1995, B_1996.BUDGET AS B_1996
FROM DEPARTMENT D
LEFT JOIN DEPT_YEAR_BUDGET B_1993
ON D.DEPT_NO = B_1993.DEPT_NO AND B_1993.FISCAL_YEAR = 1993
LEFT JOIN DEPT_YEAR_BUDGET B_1994
ON D.DEPT_NO = B_1994.DEPT_NO AND B_1994.FISCAL_YEAR = 1994
LEFT JOIN DEPT_YEAR_BUDGET B_1995
ON D.DEPT_NO = B_1995.DEPT_NO AND B_1995.FISCAL_YEAR = 1995
LEFT JOIN DEPT_YEAR_BUDGET B_1996
ON D.DEPT_NO = B_1996.DEPT_NO AND B_1996.FISCAL_YEAR = 1996
WHERE EXISTS (SELECT * FROM PROJ_DEPT_BUDGET B WHERE D.DEPT_NO = B.DEPT_NO)
21.
COMMON TABLE EXPRESSIONS
Example of simple ( non recursive ) table expressions
22.
23.
24.
25.
COMMON TABLE EXPRESSIONS
WITH RECURSIVE
R_TREE (ID, LEVEL, PATH) AS
(
SELECT ID, 0, CAST(ID AS VARCHAR(255))
FROM TREE
WHERE PARENT_ID IS NULL
UNION ALL
SELECT TREE.ID, R_TREE.LEVEL + 1,
R_TREE.PATH || ‘.’ ||
CAST(TREE.ID AS VARCHAR(8))
FROM TREE, R_TREE
WHERE TREE.PARENT_ID = R_TREE.ID
AND R_TREE.LEVEL < 10
)
SELECT TREE.*, R_TREE.LEVEL, R_TREE.PATH
FROM TREE, R_TREE
WHERE TREE.ID = R_TREE.ID
Examples of recursive table expressions
26.
COMMON TABLE EXPRESSIONS
WITH RECURSIVE
FAMILY AS
(
SELECT PEOPLE, FATHER, MOTHER
FROM PEOPLES
WHERE NAME = :CHILD
UNION ALL
SELECT F.PEOPLE_ID, F.FATHER, F.MOTHER
FROM PEOPLES F, FAMILY
WHERE F.PEOPLE_ID = FAMILY .FATHER
UNION ALL
SELECT M.PEOPLE_ID, M.FATHER, M.MOTHER
FROM PEOPLES M, FAMILY
WHERE M.PEOPLE_ID = FAMILY .MOTHER
)
SELECT * FROM FAMILY
CREATE TABLE PEOPLES
(
PEOPLE_ID INT NOT NULL PRIMARY KEY,
NAME VARCHAR(255),
FATHER INT REFERENCES PEOPLES,
MOTHER INT REFERENCES PEOPLES
)
Examples of recursive table expressions
27.
COMMON TABLE EXPRESSIONS
WITH RECURSIVE
DEPT_YEAR_BUDGET AS
(
SELECT FISCAL_YEAR, DEPT_NO,
SUM(PROJECTED_BUDGET) AS BUDGET
FROM PROJ_DEPT_BUDGET
GROUP BY FISCAL_YEAR, DEPT_NO
),
DEPT_TREE AS
(
SELECT DEPT_NO, HEAD_DEPT, DEPARTMENT,
CAST('' AS VARCHAR(255)) AS INDENT
FROM DEPARTMENT
WHERE HEAD_DEPT IS NULL
UNION ALL
SELECT D.DEPT_NO, D.HEAD_DEPT, D.DEPARTMENT,
H.INDENT || ' '
FROM DEPARTMENT D JOIN DEPT_TREE H
ON D.HEAD_DEPT = H.DEPT_NO
)
Examples of recursive table expressions
SELECT D.DEPT_NO,
D.INDENT || D.DEPARTMENT AS DEPARTMENT,
B_1993.BUDGET AS B_1993, B_1994.BUDGET AS B_1994,
B_1995.BUDGET AS B_1995, B_1996.BUDGET AS B_1996
FROM DEPT_TREE D
LEFT JOIN DEPT_YEAR_BUDGET B_1993
ON D.DEPT_NO = B_1993.DEPT_NO
AND B_1993.FISCAL_YEAR = 1993
LEFT JOIN DEPT_YEAR_BUDGET B_1994
ON D.DEPT_NO = B_1994.DEPT_NO
AND B_1994.FISCAL_YEAR = 1994
LEFT JOIN DEPT_YEAR_BUDGET B_1995
ON D.DEPT_NO = B_1995.DEPT_NO
AND B_1995.FISCAL_YEAR = 1995
LEFT JOIN DEPT_YEAR_BUDGET B_1996
ON D.DEPT_NO = B_1996.DEPT_NO
AND B_1996.FISCAL_YEAR = 1996
28.
COMMON TABLE EXPRESSIONS
Examples of recursive table expressions
29.
30.
31.
DOMAINS EVERYWHERE
SQL> SET TERM ^ ;
SQL>
SQL> CREATE DOMAIN COLOUR VARCHAR(8)
CON> DEFAULT 'RED'
CON> CHECK (VALUE IN ('RED', 'GREEN', 'BLUE'))^
SQL>
SQL> EXECUTE BLOCK
CON> RETURNS (C1 COLOUR, C2 TYPE OF COLOUR)
CON> AS
CON> BEGIN
CON> SUSPEND;
CON> END ^
C1 C2
======== ========
RED <null>
Example : domain's DEFAULT value
32.
DOMAINS EVERYWHERE
SQL> CREATE OR ALTER PROCEDURE COLOUR_USING
CON> AS
CON> DECLARE C1 COLOUR;
CON> DECLARE C2 TYPE OF COLOUR;
CON> BEGIN
CON> C2 = 'BLACK';
CON> C1 = 'WHITE';
CON> END ^
SQL>
SQL> EXECUTE PROCEDURE COLOUR_USING ^
Statement failed, SQLCODE = -625
validation error for variable C1, value "WHITE"
-At procedure 'COLOUR_USING' line: 7, col: 3
SQL>
Example : domain's CHECK CONSTRAINT
33.
34.
35.
36.
37.
MERGE
CREATE TABLE AMOUNTS
(
GOODID INT,
SUMMA NUMERIC(18, 4),
CONSTRAINT PK_AMOUNTS
PRIMARY KEY (GOODID)
) ;
CREATE TABLE ENTRIES
(
GOODID INT,
DT DATE,
DBT NUMERIC(18, 4),
CRD NUMERIC(18, 4),
CONSTRAINT PK_ENTRIES
PRIMARY KEY (GOODID, DT)
) ;
MERGE INTO AMOUNTS USING ENTRIES E
ON GOODID = E.GOODID
WHEN MATCHED
THEN UPDATE SET SUMMA = SUMMA + E.DBT - E.CRD
WHEN NOT MATCHED
THEN INSERT (GOODID, SUMMA)
VALUES (E.GOODID, E.DBT - E.CRD)
MERGE example
38.
39.
Questions ?
Notas do Editor
- CONNECT
Database connection established
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are returned to the client
The transaction is committed
- DISCONNECT
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are swallowed
The transaction is committed
The attachment is disconnected
- TRANSACTION START
Triggers are fired in the newly user created transaction – uncaught exceptions are returned to the client and the transaction is rolled-back.
- TRANSACTION COMMIT
Triggers are fired in the committing transaction (before actual commit) - uncaught exceptions rollback the triggers savepoint, the commit command is aborted and the exception is returned to the client.
Note: for two-phase transactions the triggers are fired in &quot;prepare&quot; and not in commit.
- TRANSACTION ROLLBACK
Triggers are fired in the rolling-back transaction (before actual rollback) - changes done will be rolled-back togheter with the transaction and exceptions are swallowed
gfix and gstat connections does not fire database triggers
- CONNECT
Database connection established
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are returned to the client
The transaction is committed
- DISCONNECT
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are swallowed
The transaction is committed
The attachment is disconnected
- TRANSACTION START
Triggers are fired in the newly user created transaction – uncaught exceptions are returned to the client and the transaction is rolled-back.
- TRANSACTION COMMIT
Triggers are fired in the committing transaction (before actual commit) - uncaught exceptions rollback the triggers savepoint, the commit command is aborted and the exception is returned to the client.
Note: for two-phase transactions the triggers are fired in &quot;prepare&quot; and not in commit.
- TRANSACTION ROLLBACK
Triggers are fired in the rolling-back transaction (before actual rollback) - changes done will be rolled-back togheter with the transaction and exceptions are swallowed
gfix and gstat connections does not fire database triggers
- CONNECT
Database connection established
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are returned to the client
The transaction is committed
- DISCONNECT
A transaction is started
Triggers are fired - uncaught exceptions rollback the transaction, disconnect the attachment and are swallowed
The transaction is committed
The attachment is disconnected
- TRANSACTION START
Triggers are fired in the newly user created transaction – uncaught exceptions are returned to the client and the transaction is rolled-back.
- TRANSACTION COMMIT
Triggers are fired in the committing transaction (before actual commit) - uncaught exceptions rollback the triggers savepoint, the commit command is aborted and the exception is returned to the client.
Note: for two-phase transactions the triggers are fired in &quot;prepare&quot; and not in commit.
- TRANSACTION ROLLBACK
Triggers are fired in the rolling-back transaction (before actual rollback) - changes done will be rolled-back togheter with the transaction and exceptions are swallowed
gfix and gstat connections does not fire database triggers
Intro
Sometimes programmers have necessity for the data with temporary nature. For example - create a data set, work with it and remove it. If it is necessity is seldom or in general once then it is possible to create and drop the table. But what to do if such tasks arises frequently ? Constant creation and dropping of tables at first is not good from the security point of view (end users usual have no such rights), at second it is extremely not recommended to perform DDL on production database and, at third, productivity of such approach is far from perfect. Besides if the own set of temporary data are needed for several users simultaneously, they will be compelled to assign them unique names, and it means that stored procedures can't work with such tables.
One of the widespread approaches to get rid of these problems is to create the permanent table with an additional service field populated with CURRENT_USER or with the private value of generator used during data sets lifetime. In spite of the fact that this approach is free from lacks of previous one it also has its own lacks. In particular because of frequent deletions many record versions will be accumulated which then should be removed from database. Since usually additional field is indexed and included in all queries to such tables the removed records are never visited again and this garbage will sit in DB until the sweep runs.
Fortunately the SQL standard has the solution of such problems (yes it not always prevents quiet life of the ordinary programmer :). This solution is a various kinds of temporary tables. Today we consider global temporary tables as they were implemented in IB 7.5 first, then it was included in Fyracle and now they are in Firebird also
Definition
Global temporary tables (GTTs) are tables with permanent metadata, stored in the system catalogue, but with the temporary data. GTT's may be of two kinds - with the data, persistent within lifetime of connection in which the given GTT was referenced, and with the data, persistent within only during lifetime of referencing transaction. The data from different connections (transactions) are isolated from each other, but metadata of the global temporary table are shared between all connections and transactions.
All kinds of constraints between temporary and persistent tables follow the rule below:
a) references between persistent and temporary tables are forbidden
b) GTT with ON COMMIT PRESERVE ROWS can't have reference on GTT with ON COMMIT DELETE ROWS
c) Domain constraints can't have reference on GTT.
This is a table for easier understanding of which references are allowed
Implementation details:
GTT instance (set of data rows created by and visible within given connection or transaction) created when it first referenced, usually at statement prepare time. Each instance has its own private set of pages on which data and indexes are stored. Data rows and indexes have the same physical storage layout as permanent tables.
When connection or transaction ends all pages of an GTT instance are released immediately (this is similar as when you do DROP TABLE but metatada remains in database of course). This is much quicker than traditional row by row delete + garbage collection of deleted record versions. Each data (index) page contains rows (keys) from the same GTT instance therefore connection ransaction should readwrite less pages than if an GTT instances were isolated by the service field. Note that with “service field” design data pages will contain rows also from old dead connections or transactions until it will be garbage collected.
Sometimes our design required more space than “service field” design. For example 10 instances of GTT each with one row occupied 10 data pages, while “service field” design needs only one page (if it can fit 10 rows). But this difference visible only with big amount of small tables and we assume it reasonable price for benefits (speed and near instant cleanup).
In Fyracle data pages of all of the GTT’s instances placed in the database file. In Firebird 2.1 this pages placed in separate temporary file’s – this allows to put temporary data into another hard disk and reach more performance level. Each connection have its own temporary file created when this connection first referenced some GTT. Also this temporary files always opened with “Forced writes = OFF” settings despite of database setting.
There are no limit on number of GTT instances. If you have N transactions active simultaneously and each transaction have referenced some GTT then you’ll have N GTT’s instances.
CTE is standard feature. It is supported in modern versions of DB2, ORACLE, MSSQL, ASA. In FB history it was first implemented in Fyracle by Paul Ruizendaal to support Oracles non standart syntax of recursive queries.
CTE allows to write very complex queries in simple, natural manner. All you need is just decompose complex query by simple parts (as you often do with complex procedures in such programming language as C, PASCAL etc), assign names to this parts and use it in near any possible combinations.
CTE is like view’s, locally defined within main query. Or we can compare it with local procedures in pascal language. From the engine POV CTE is a derived table so no intermediate materialization is performed
WITH can’t be nested but this is not a strict limitation as you always can reference one CTE from another
Here the simple example of how we can build cross-tab report about per year budget of each department using one CTE several times in the same query. Our standard “employee” database is used in this example (someone can even remember it :)
We defined CTE DEPT_YEAR_BUDGET which sums per-year budget of each department. Then we join it four times with DEPARTMENT table to build our simple cross-tab. Also we exclude from report departments which have no budget at all
This example shows that we can reuse same CTE several times in query
here the results of query above.
Well, CTE is a powerful mechanism to create complex, powerful query. But that’s not all.
With recursive CTE we can create even recursive queries now.
How it work:
engine start execution from non-recursive members
for each evaluated row engine start execution of each recursive member using current row values as parameters
if current instance of recursive member produced no rows engine returns one step back and get next row from previous resultset
Memory and CPU overhead of recursive CTE is much less than overhead of recursive stored procedures
currently recursion depth is limited by hardcoded value of 1024
it will be discussed later about implement non standard optional syntax enhancement to allow user specify maximum recursion level immediately in query
Aggregates limitation come from standard. I have no clear explanation of it
This example shows how to solve several classical tasks when working with recursive data structures
query returns tree-like table in order of tree walk. it is very useful if client application can’t itself show tree-like structure
we see how easy nesting level can be evaluated
we see how we can limit number of processed levels of tree
we also return full path from the root to the current tree node – it can be used by client application as a hint to end-user
this example is more complex. it shows how two recursive members is used to return two recursion path
very simple table PEOPLE contains ID of people, his name and ID’s of his parents
query returns whole genealogical tree of given people - i.e. his parents, parents of his parents and so on
also we see that parameters can be used inside of CTE also (if someone think it is allowed in main query only)
return to the our example of use of simple CTE and include department’s as a tree into that cross-tab report
let define CTE DEPT_TREE which shows departments in tree-like style and replace in main query plain table DEPARTMENT by this recursive CTE
To let our report looks as real tree I included INDENT field into our recursive CTE DEPT_TREE. More nesting level – more indentation of name
Easy is not is ?
Note - error message contains variable name and linecolumn numbers