1. Arthur Luz | SQL Server MCSA & MCT
arthurjosemberg@gmail.com
http://arthurluz.wordpress.com
2. Who am I?
Data Insights Consultant at Microsoft
Technical Writer at Data’s Light Blog
MCSA e MCT em SQL Server
Official Instructor at Hepta Novintec
3. Overview – SQL Server 2016 Temporal Tables
New Clauses – Data Definition
New Clauses – Data Manipulation
Considerations for resource use
Main Differences - CDC and Temporal Tables
Usage Scenarios
Feature Improvement
Schedule for today
6. Overview – History Tables
Auditing all data changes and performing data
forensics when necessary
Reconstructing state of the data as of any time in the
past
Calculating trends over time – Anomaly Detection
Maintaining a slowly changing dimension for
decision support applications
Recovering from accidental data changes and
application errors
7. New clauses – Data Definition
CREATE TABLE Department (
DeptID int NOT NULL PRIMARY KEY CLUSTERED ,
DeptName varchar(50) NOT NULL ,
ManagerID INT NULL ,
ParentDeptID int NULL ,
SysStartTime datetime2 GENERATED ALWAYS AS ROW START NOT NULL ,
SysEndTime datetime2 GENERATED ALWAYS AS ROW END NOT NULL ,
PERIOD FOR SYSTEM_TIME (SysStartTime, SysEndTime)
) WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.DepartmentHistory) ) ;
GENERATED ALWAYS AS ROW START –
Determina que essa coluna recebá do sistema a
data inicial de validade do registro
GENERATED ALWAYS AS ROW END –
Determina que essa colina receberá do sistema a
data final de validade do registro.
PERIOD FOR SYSTEM_TIME – Realiza a
organização histórica do registro de acordo com
as datas de inicio e fim de validade da tupla.
SYSTEM_VERSIONING = ON – Habilita a
feature para a tabela que está sendo criada ou
alterada.
8. BEGIN TRANSACTION
-- Add required period columns and designation
ALTER TABLE dbo.Person ADD
DateTimeStart DATETIME2(0) GENERATED ALWAYS AS ROW START NOT NULL
CONSTRAINT DFT_person_datetimeStart DEFAULT('19000101 00:00:00'),
DateTimeEnd DATETIME2(0) GENERATED ALWAYS AS ROW END NOT NULL
CONSTRAINT DFT_person_datetimeEnd DEFAULT('99991231 23:59:59'),
PERIOD FOR SYSTEM_TIME (DateTimeStart, DateTimeEnd);
-- Remove temporary DEFAULT constraints
ALTER TABLE dbo.Person
DROP CONSTRAINT DFT_person_datetimeStart, DFT_person_datetimeEnd;
-- Turn system versioning on
ALTER TABLE dbo.Person
SET ( SYSTEM_VERSIONING = ON ( HISTORY_TABLE = dbo.PersonHistory ) );
COMMIT TRANSACTION;
10. New clauses – Data Manipulation
DECLARE @datetime AS DATETIME2 = '2016-11-08 20:00:00'
SELECT *
FROM dbo.Person FOR SYSTEM_TIME
AS OF @datetime
ORDER BY DateTimeStart ASC,
DateTimeEnd ASC
Clauses SYSTEM_TIME
allow to realize temporal
queries transparently for
user and applications.
11. FOR SYSTEM_TIME AS OF @datetime
Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)
sysEnd > @datetime
sysStart <= @datetime
New clauses – Data Manipulation
Returns a table with a rows containing the
values that were actual (current) at the
specified point in time in the past.
12. FOR SYSTEM_TIME
FROM @Start TO @End
Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)
sysStart < @End
sysEnd > @Start
New clauses – Data Manipulation
Returns a table with the values for all row
versions that were active within the specified
time range, regardless of whether they started
being active before the <start_date_time>
parameter value for the FROM argument or
ceased being active after the <end_date_time>
parameter value for the TO argument.
13. FOR SYSTEM_TIME
BETWEEN @Start AND @End
Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)
sysStart <= @End
sysEnd > @Start
New clauses – Data Manipulation
Same as above in the FOR SYSTEM_TIME
FROM <start_date_time> TO <end_date_time>
description, except the table of rows returned
includes rows that became active on the upper
boundary defined by the <end_date_time>
endpoint.
14. FOR SYSTEM_TIME
CONTAINED IN (@Start,@End)
Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)
sysStart >= @Start
sysEnd <= @End
New clauses – Data Manipulation
Returns a table with the values for all row
versions that were opened and closed within the
specified time range defined by the two
datetime values for the CONTAINED IN
argument.
16. Considerations for resource use
A temporal table must have a primary key defined in order to correlate records between the current table and
the history table, and the history table cannot have a primary key defined.
The SYSTEM_TIME period columns used to record the SysStartTime and SysEndTime values must be
defined with a datatype of datetime2.
If the name of a history table is specified during history table creation, you must specify the schema and table
name.
By default, the history table is PAGE compressed.
If current table is partitioned, the history table is created on default file group because partitioning
configuration is not replicated automatically from the current table to the history table.
Temporal and history tables cannot be FILETABLE and can contain columns of any supported datatype other
than FILESTREAM since FILETABLE and FILESTREAM allow data manipulation outside of SQL Server and
thus system versioning cannot be guaranteed.
17. Considerations for resource use
While temporal tables support blob data types, such as (n)varchar(max), varbinary(max), (n)text, and image,
their will incur significant storage costs and have performance implications due to their size. As such, when
designing your system, care should be taken when using these data types.
History table must be created in the same database as the current table. Temporal querying over Linked Server
is not supported.
History table cannot have constraints (primary key, foreign key, table or column constraints).
Indexed views are not supported on top of temporal queries (queries that use FOR SYSTEM_TIME clause)
Online option (WITH (ONLINE = ON) has no effect on ALTER TABLE ALTER COLUMN in case of system-
versioned temporal table. ALTER column is not performed as online regardless of which value was specified for
ONLINE option.
INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns. Attempts to insert
values directly into these columns will be blocked.
18. Considerations for resource use
TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON
Direct modification of the data in a history table is not permitted.
ON DELETE CASCADE and ON UPDATE CASCADE are not permitted on the current table. In other words,
when temporal table is referencing table in the foreign key relationship (corresponding to parent_object_id in
sys.foreign_keys) CASCADE options are not allowed. To work around this limitation, use application logic or
after triggers to maintain consistency on delete in primary key table (corresponding to referenced_object_id in
sys.foreign_keys). If primary key table is temporal and referencing table is non-temporal, there’s no such
limitation.
INSTEAD OF triggers are not permitted on either the current or the history table and AFTER triggers are
permitted only on the current table.
Regular queries only affect data in the current table. To query data in the history table, you must use temporal
queries.
19. Considerations for resource use
An optimal indexing strategy will include a clustered columns store index
and / or a B-tree rowstore index on the current table and a clustered
columnstore index on the history table for optimal storage size and
performance. If you create / use your own history table, we strongly
recommend that you create this type of index consisting of period columns
starting with the end of period column to speed up temporal querying as
well as speeding up the queries that are part of the data consistency check.
The default history table has a clustered rowstore index created for you
based on the period columns (end, start). At a minimum, a non-clustered
rowstore index is recommended.
20. Table 1
Main Differences – CDC x Temporal Tables
Log
_CDC 1
Uses all table columns.
Possible to select specific columns.
Works synchronously.
Works asynchronously. Allows change in table
when feature is active.
It does not allow change in the
table when the feature is active.
Used to maintain history in the
transactional system.
Used only for momentary
capture of history.
Simple queries that are transparent
to users and applications.
Complex queries that require
in-depth knowledge.
Version INSERTEs, UPDATEs and DELETEs.
Version only UPDATEs and DELETEs.
22. Usage Scenarios
Data Audit
Point in Time
Analysis
(Time Travel)
Calculating trends
over time
(Anomaly Detection)
Slowly-Changing
Dimensions
Repairing Row-Level
Data Corruption
25. Feature Improvement – In-Memory
Only durable memory-optimized tables can be system-versioned
(DURABILITY = SCHEMA_AND_DATA).
Queries that affect only the current table (in-memory) can be used in natively
compiled T-SQL modules. Temporal queries using the FOR SYSTEM TIME
clause are not supported in natively compiled modules. Use of the FOR
SYSTEM TIME clause with memory-optimized tables in ad hoc queries and
non-native modules is supported.
When SYSTEM_VERSIONING = ON, an internal memory-optimized staging
table is automatically created to accept the most recent system-versioned
changes that are results of update and delete operations on memory-
optimized current table.
Data from the internal memory-optimized staging table is regularly moved to
the disk-based history table by the asynchronous data flush task. This data
flush mechanism has a goal to keep the internal memory buffers at less than
10% of the memory consumption of their parent objects.
Hybrid Transactional/Analytical Processing (HTAP)
Lembrar que o versionamento dos registros s’ao realizados com ‘sysdatetimeoffset’
Retorna uma tabela com os valores de todas as versões de linhas que estavam ativas dentro do intervalo de tempo especificado, independentemente de terem começado a ser ativas antes do valor do parâmetro <start_date_time> para o argumento FROM ou terem deixado de estar ativas após o valor do parâmetro <end_date_time> TO.
O mesmo que acima na descrição do FOR SYSTEM_TIME FROM <start_date_time> TO <end_date_time>, exceto que a tabela de linhas retornada inclui linhas que se tornaram ativas no limite superior definido pelo nó de extremidade <end_date_time>.
Retorna uma tabela com os valores para todas as versões de linha que foram abertas e fechadas dentro do intervalo de tempo especificado definido pelos dois valores de data e hora para o argumento CONTAINED IN.
Do not perform massive deletes from current table in order to increase available RAM by cleaning up the space. Consider deleting data in multiple batches with manually invoked data flush in between by invoking sp_xtp_flush_temporal_history, or while SYSTEM_VERSIONING = OFF.
Do not perform massive table updates at once as it can result in memory consumption that is twice the amount of memory required to update a non-termporal memory-optimized table. Doubled memory consumption is temporary because data flush task works regularly to keep memory consumption of internal staging table within projected boundaries in the steady state (around 10% of memory consumption of current temporal table). Consider doing massive updates in multiple batches or while SYSTEM_VERSIONING = OFF, such as using updates to set the defaults for newly added columns.