This presentation by Paul Bertucci describes an ordered method of determining what users need and which SQL Server data distribution solution is best to use. There are many needs of data throughout an organization. Getting data to those who need it can be accomplished many different ways with SQL Server 2008 technologies. This presentation covers data replication, database mirroring and snapshots, older methods such as log shipping and linked servers, and new methods such as using the sync framework. You'll Learn * Each of SQL Server’s main data distribution solutions * How to determine which solution to use to solve different purposes

1. Distributing Data Techniques with
Distributing Data Techniques with
Distributing Data Techniques with
SQL Server 2008 R2
Q
Speaker: Paul Bertucci
Founder, Database Architechs
d b hi h
Silicon Valley SQL Server User Group
October 19, 2010
Mark Ginnebaugh, User Group Leader,
mark@designmind.com

2. Distributing Data Techniques
Distributing Data Techniques
with SQL Server 2008 R2
Copyright 2010 All Rights Reserved

3. Agenda
g
• Determining where data is needed
• What SQL Server options do you have
• How to get it there
• Work within your constraints
 Security
 Regulatory (like PII…)
 Administratively
Ad i i t ti l
 Data volatility/sizes
 SLA’s you must deliver
 Disaster Recovery/Business Continuity
 Skills/Knowledge of your team
 Resources (like $$’s, so on)

4. Reasons to Distribute Data
Reasons to Distribute Data

5. SQL Server 2008 SQL Server 2008
Reporting/ODS
Primary Reporting
OLTP Server
19FIG02
OLTP DB Rpt DB
pt
SQL Server 2008
Enabling/Partitioning
Europe
Server
SQL Server 2008
USA xyz DB
SQL Server 2008
(Headquarters)
Asia
xyz DB Server
xyz DB
SQL Server 2008 SQL Server 2008
North America Regionalization Europe
Region Region
xyz DB
y (multiple owners) xyz DB
y
SQL Server 2008 SQL Server 2008
Primary Failover Hot Spare
(Fail-over)
xyz DB xyz DB

6. Distributing periodic updates to
Data Marts from a “master” Data Warehouse
Data
Mart
Master
SQL Server Data Warehouse
2008
Data
Mart
Data
Mart
SQL Server
2000
SQL Server
2005

7. Enabling a Data Warehouse
from Legacy Systems
ORACLE
Master
Data Warehouse
SQL Server
2005
SQL Server
2008

8. Initialization of tables Heterogeneous
for a heterogeneous Subscriber Subscriber
in Replication
ORACLE
Publisher
Subscriber
SQL Server
2008
SQL Server
2008
Subscriber
SQL Server
2005

9. Pulling in Sales Order information
Sales Force Automation from disparate sources
Application for import into SQL Server 2008
Access
SQL Server
Excel 2008

10. Data Mart
OLTP
Database SQL Server
2008
SQL Server
2008 Excel
Export from the OLTP Database
to a business intelligence
Data Mart & Excel

11. Connection
Results Results
SQL Server SQL Server
2008 2008
Client Local Server Remote Server

12. Decision‐Tree Path Traversal
Database
Mirroring
Integration
I t ti With Snapshots
S h t
Services
1a2c3 1a2d3 Detach/
1a2e3 Attach
BCP 1a2b3
b c d
a 1a2 e
1a2a3
1 2 3
a b 1b2
1e2 Log
Sync e 1 c
d Shipping
Framework 1c2
1d2
Backup/ Data
Restore Replication
Linked
Servers

13. Data Log Database Mirroring Linked Backup/ Detach/ Sync
SSIS BCP
Replication Shipping with Snapshots Servers Restore Attach Framework
Large
Data
High
Volatility
On Line
Many Targets
Low
Latency
Transformation/
Filtering

14. Data Replication
Data Replication

15. Central Publisher
SQL Server SQL Server
2008 (default option) 2005
Adventure Adventure
Works2008 Works2008
translog SQL Server translog
Subscription 2008 Subscription
Server 19FIG10 Server
Adventure
Works2008
translog
Publication
Server
distribution SQL Server
2000
Oracle Distribution
Server Adventure
Adventure Works2008
Works2008 translog
Subscription Subscription
Server Server
15

16. Central Publisher
SQL Server
19FIG11 Remote Distributor SQL Server
2008 2005
Adventure Adventure
Works2008 Works2008
SQL Server
translog 2008 translog
Subscription Subscription
Server Adventure Server
Works2008
translog
Publication
Server
SQL Server
2008 SQL Server
2000
Oracle distribution
Remote Distribution Adventure
Adventure Works2008
Server
Works2008
translog
Subscription Subscription
Server Server

17. Publishing Subscriber
SQL Server SQL Server
2008 2005
SQL Server
Publication
2008
Adventure Adventure
Server
Works2008 Works2008
Adventure
19FIG12
Works2008 translog
S
translog
Subscription translog Subscription
Distribution
Server Server
Server
distribution
Subscription
SQL Server
Serveer
2008
Adventure SQL Server
Works2008 2000
tion
Oracle
er
translog
Publicat
Serve
Adventure
Adventure Works2008
Works2008
translog
Subscription
ion
Subscription
Distributi
Server r
Server Server
distribution
17

18. Publication SQL Server SQL Server
tion
2008 2008
19FIG13
er
er
Central
Serve
Publicat
Serve
Adventure Adventure
Works2008 Works2008
translog
Subscriber translog
ion
Distribution
Distributi
Server r
Server
distribution distribution
SQL Server 2008
Subscription
Server
Adventure
Works2008
SQL Server SQL Server
Publication
ation
2008 2008
ver
ver
Publica
Serv
Serv
Adventure Adventure
Works2008 Works2008
translog translog
ution
ution
er
er
Serve
Serve
Distribu
Distribu
distribution distribution

19. Multiple Publishers
of a Single Table
19FIG14Server
on
on
Subscriptio
Subscriptio
SQL S SQL Server
S
Server
Server
2008 2008
Adventure Adventure
on
on
Works2008 Works2008
Publicatio
Publicatio
Serverr
Server
translog translog
Distribution
Distribution
Server
Server
distribution distribution
Territory 1 Territory 2
Customers Table
CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate
1345 1 AW1345 I X69G9.. 120203
1356 2 AW1356 I W211G.. 051605
2354 1 AW2354 S 7SQ78K.. 106705
3346 2 AW3346 I W12DV.. 022305
7643 2 AW7643 S WZ8R4.. 122205
7901 2 AW7901 I S2345X.. 041506
8921 1 AW8921 I RT66Y.. 031206

20. SQL Server
Updating
U d ti
blication
2008 Subscriber
Server
19FIG15
Adventure
Works2008
Pub
S
translog SQL Server Queue
stribution
(Queue Reader Agent)
Server
S
distribution
Dis
Update Customers
set AccountNumber = null
where CustomerID = 12345
SQL Server SQL Server
2008 2008
Adventure Adventure
Works2008 Works2008
W k 2008
translog translog
Subscription Subscription
Server Server

21. SQL Server SQL Server
2008 2008
19FIG16
Adventure
Works2008 Peer-to-Peer Adventure
Works2008
translog translog
Publication Publication
Server Server
North American
o e ca Asia
sa
SQL Server 2008
Server Server
distribution
Peer Originator id: 1 Distribution Server Peer Originator id: 2
Customers Table
CustomerID
C t ID TerritoryID
T it ID AccountNumber C
A tN b CustomerType
T rowguid
id ModifiedDate
M difi dD t
1345 1 AW1345 I X69G9.. 120203
1356 2 AW1356 I W211G.. 051605
2354 1 AW2354 S 7SQ78K.. 106705
3346 2 AW3346 I W12DV..
W12DV 022305
7643 3 AW7643 S WZ8R4.. 122205
7901 5 AW7901 I S2345X.. 041506
8921 4 AW8921 I RT66Y.. 031206

22. Publisher
Publication
P bli ti
AW_Vertical
Adventure
Works2008
translog CustomerV (Article)19FIG05
SQL Server
2008
CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate CustomerID TerritoryID CustomerType
1345 1 AW1345 I X69G9.. 120203 1345 1 I
1356 2 AW1356 I W211G.. 051605 1356 2 I
2354 1 AW2354 S 7SQ78K.. 106705 2354 1 S
3346 2 AW3346 I W12DV.. 022305 3346 2 I
7643 3 AW7643 S WZ8R4.. 122205 7643 3 S
7901 5 AW7901 I S2345X.. 041506 7901 5 I
8921 4 AW8921 I RT66Y.. 031206 8921 4 I

23. Publisher
Publication
AW_Horizontal
Adventure
Works2008
19FIG06
CustomerH (Article)
translog
SQL Server
2008
CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate
1345 1 AW1345 I X69G9.. 120203
1356 2 AW1356 I W211G.. 051605
2354 1 AW2354 S 7SQ78K.. 106705
3346 2 AW3346 I W12DV.. 022305 Only these Rows!
7643 3 AW7643 S WZ8R4.. 122205
7901 5 AW7901 I S2345X.. 041506
8921 4 AW8921 I RT66Y.. 031206
CustomerID TerritoryID AccountNumber CustomerType
yp rowguid
g ModifiedDate
2354 1 AW2354 S 7SQ78K.. 106705
7643 3 AW7643 S WZ8R4.. 122205

24. Publisher
Publication
Adventure
Ad t
19FIG07
AW_H_and_V
Works2008 07FIG08
translog CustomerHV (Article)
SQL Server
2008
CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate
1345 1 AW1345 I X69G9.. 120203
1356 2 AW1356 I W211G.. 051605 Only these Columns
2354 1 AW2354 S 7SQ78K.. 106705 and
3346 2 AW3346 I W12DV.. 022305 these Rows!
7643 3 AW7643 S WZ8R4.. 122205
7901 5 AW7901 I S2345X.. 041506
8921 4 AW8921 I RT66Y.. 031206
CustomerID TerritoryID
y CustomerType
2354 1 S
7643 3 S

25. AW_H_and_J (Publication)
CustomerH (Article)
19FIG08
07FIG09
OrdersHJ (Article)
CustomerID TerritoryID AccountNumber CustomerType rowguid ModifiedDate Only the orders
1345 1 AW1345 I X69G9..
X69G9 120203 for
1356 2 AW1356 I W211G.. 051605
customer
2354 1 AW2354 S 7SQ78K.. 106705 CustomerID
2354
type “S”
3346 2 AW3346 I W12DV.. 022305
7643 3 AW7643 S WZ8R4..
WZ8R4 122205 7643
7901 5 AW7901 I S2345X.. 041506
8921 4 AW8921 I RT66Y.. 031206
join
SalesOrderID RevisionNumber OrderDate DueDate Status CustomerID
43659 1 7/1/2001 7/13/2001 5 1345
43660 1 7/1/2001 7/13/2001 5 1356
43661 1 7/1/2001 7/13/2001 5 2354
SalesOrderID OrderDate CustomerID
43662 2 7/1/2001 7/13/2001 5 3346
43661 7/1/2001 2354
43663 1 7/1/2001 7/13/2001 5 7643
43664 1 7/1/2001 7/13/2001 5 7901 43663 7/1/2001 7643
43665 1 7/1/2001 7/13/2001 5 8921 25

26. Publisher
AW (Publication)
(P bli ti ) Subscriber
SQL Server
cation
19FIG17 2008 Customers (Article)
rver
SQL Server
Public
Adventure 2008
Ser
Works2008
Adventure
translog Works2008
bution
PUSH
rver
translog
Distrib
Ser
distribution
di t ib ti
Subscription
Server
PUSH: initiated by Publisher PULL
(agent will be on the Publisher if local distribution,
or on the Distributor if remote distribution)
PULL: initiated by Subscriber
y
(agent will be on the subscriber)

27. Data Access Latency
Distributing Data
Distributing Data
Autonomy
Sites
(locations)
Frequency Network Machines Owner Other REPLICATION
Read Only Each site only Central Publisher
fast/ n Transactional repl
Reporting short high many high 1 OLTP needs regional
stable server/site site data filter by region
Read Only Each site only Central Publisher
fast/ n Snapshot repl
Reporting long high many low 1 OLTP needs regional
stable server/site site data filter by region
Read Mostly
ead ost y Regional updates Central Publisher
g p
fast/ n Transactional repl
A few updates short high < 10 medium 1 OLTP on one table
stable server/site site Updating Subs
Read Mostly Regional update Central Publisher
slow/ n All Merge repl
A few updates medium high < 10 medium all tables
unreliab server/site update
Read equal Regional update Peer-to-Peer
fast/ 1 All Transactional
Equal updates short high < 10 medium all tables
stable server/site update repl
Inserts 1 Each site only Central Subscriber
fast/ 1 Transactional repl
(new orders) short high many high needs regional
stable server/site report data
site
Central Publisher
Hot/Warm
/ Very high <2 high fast/
f / 1 Fail-over Remote Distributor
1 OLTP
Spare short stable server/site site Transactional repl

28. Log Shipping
Yes, it is still here..
,

29. Log Shipping
“Source”
SQL Server 2008
“Destination”
SQL Server 2008
Q
TxnLog
T L
Primary CallOne DB backups Secondary
Server Server
translog
CallOne DB
BackupCallOne_tlog_200405141120.TRN
pped
wer
LogShareCallOne_tlog_200405141120.TRN
LogShareCallOne tlog 200405141120 TRN TxnLog
Last log ship
Delay Answ
TxnLog
T L
Copies Restores
“Monitor”
SQL Server 2008
S Delay between
logs loaded
Delay Answer
Monitor
Server MSDB DB
Don’t build on this, it is being deprecated !

30. Database Mirroring
with Snapshots
with Snapshots

31. Client Client Client Client
Network
A
SQL Server 2008 SQL Server 2008
Mirror Server
Mi S
DB Mirroring
Adventure D
Principal Adventure D
Works DB
Server B Works DB
translog
t l
translog
C
SQL Server xyz
Witness
t ess
Server MSDB DB
31

32. SQL Server 2008 SQL Server 2008
Mirror Server
Principal 10
Server AdventureWorks
AdventureWorks
DB DB
translog translog
Database Database
Snapshot Snapshot
comes the mirror.
If this server node
SQL Server 2008
m
n
Witness
MSDB DB
Server
bec
t
32

33. SQL Server 2008
01AdventureWork
Source CREATE DB ‘xyz’
s DB
AS SNAPSHOT OF
Server AdventureWorks
translog
Database
D t b
Snapshot
--- Reverting --- if needed ---
RESTORE DB
AdventureWorks
FROM DATABASE_SNAPSHOT = ‘xyz’

34. Source Data
Pages
Source
02
SQL Server 2008
SQL Sparse
Server File
Pages Snapshot
AdventureWorks AdventureWorks
DB DB
Empty Sparse file
of a Snapshot just created
(no p
( updates to original
g
data pages have occurred yet)
34

35. Source
Data
Pages
Source
03
SQL Server 2008
SQL Sparse
Server File
Pages Snapshot
AdventureWorks AdventureWorks
DB DB
System Catalog Copy of original
of changed pages pages for snapshot
only when a page is changed
(Copy-on-write)
35

36. Snapshot
Source Data Users
Pages
SELECT …..data…….
FROM AdventureWorks
SNAPSHOT
04
SQL Server 2008
SQL
Server
Snapshot
AdventureWorks AdventureWorks
DB DB
System Catalog Sparse
of changed pages File
Pages
36

37. SQL Server 2008 SQL Server 2008
Mirror Server
Principal 10
Server AdventureWorks
AdventureWorks
DB DB
translog translog
Database
Database
Snapshot
Snapshot
comes the mirror.
If this server node
SQL Server 2008
m
n
Witness
MSDB DB
Server
bec
t
37

38. PH Topology With Snapshots
Real time
Reporting
R ti
Users
Network
SQL Server 2008
N
Principal
Server
SQL Server 2008
OLTP Application
n
Active Mirror Server
stered
Replication
Adventure Adventure
Works DB Works DB
Clus
translog translog
Database
Passive
PH Snapshot
Topology
Principal Network
Server
SQL Server 2008
Point in time 38
Reporting Users

39. Publisher DB Mirror and
SQL Server 2008 SQL Server 2008 Replication
Principal Mirror SQL Server 2008
Server Server
Subscriber
Distributor
s bu o
SQL Server 2008 SQL Server 2008
Principal Mirror
Server Server
SQL Server 2008
SQL Server 2008
Witness
Server
Subscriber
39

40. Integration Services

41. Excel
OLAP
Data Warehouse
Access
Multidimensional
OLTP Cube
Databases
Other Data Sources
OLTP
Databases
SQL Server
Flat
XML
Web Services files

42. Package
P k
Task Task
Task
ainer
Source
Cache
e
Conta Data
Flow Transformation
Task
Destination
D ti ti
Task Task
Task

43. SSIS
Data Mart
Join and Aggregate
e
OLTP
Database
Tas to
SQL Server
sk
AdventureWorks 2008
d
UnleashedDataMart
SQL Server
2008 Excel
.csv flat file
Export from the OLTP Database
to a business intelligence
Data Mart & Excel/.csv flat file

44. Linked Servers

45. OLE DB Provider
for MS SQL Server
SQL Server
6.5 – 2008
OLE DB Provider DLL’s
OLE DB Provider Sybase
for ODBC Informix
DB2
ODBC Others
OLE DB Provider Excel spreadsheet
for Jet Jet FoxPro Database
database Dbase file
MS OLE DB OLE DB Provider file Access Database
E
Provider
P id for Oracle
f O l
SQL Server
ORACLE
2008
OLE DB Provider
P id
for Indexing Service Active Directory Services
Index Server
Others
Client Tier Server Tier Remote Data Source Tier
(can be “Linked”)

46. Might be to easy?
Might be to easy?
-------------------------------------------------------------
-- Selecting data from Excel spreadsheet via Linked Server --
-------------------------------------------------------------
SELECT substring(CustomerID,1,5) as 'CustID', substring(ContactName,1,18) as
ContactName,
substring(City,1,10) as 'Cit ' substring(Country,1,15) as 'C
b t i (Cit 1 10) 'City', b t i (C t 1 15) 'Country''
t
FROM [Linked ExcelSW]...[SWCustomers$]
ORDER BY ContactName
g
go
CustID ContactName City Country
------ ------------------ ---------- ---------------
PICCO Adam Greifer Los Angele USA
BLAUS John Monroe Concord USA
CHOPS Martin Sommer Berkeley USA

47. Microsoft Sync Framework
Microsoft Sync Framework
A comprehensive synchronization platform that
enables collaboration and offline access for
enables collaboration and offline access for
applications, services, and devices. It features
technologies and tools that enable roaming,
sharing, and taking data offline.
By using Sync Framework, developers can build
B i S F k d l b ild
sync ecosystems that integrate any application
with any data from any store that uses any
y y y
protocol over any network. Mentioned here
because of its replication‐like behavior for
“occasionally connected” applications.
occasionally connected applications

48. Others
• BCP
• Detach/Attach
• Backup/Restore
k /

49. Copy on Write Technology

50. DB Mirroring
Client Client Client Client
Network
A
SQL Server 2008 SQL Server 2008
Mirror Server
Mi S
Principal Adventure D Adventure
Works DB D
Server B Works DB
translog
t l
translog
C
SQL Server xyz
Witness
t ess
Server MSDB DB
50

51. Source
Data
Pages
Source
03
SQL Server 2008
SQL Sparse
Server File
Pages Snapshot
AdventureWorks AdventureWorks
DB DB
System Catalog Copy of original
of changed pages pages for snapshot
only when a page is changed
(Copy-on-write)
51

52. Database Architechs
Paul Bertucci
 Founder Database Architechs
 Former Chief Data Architect – Veritas & Symantec
 Currently
C rrentl Chief Architect - A todesk
Autodesk
 Author of SQL Server 2000, 2005 & 2008/R2 Unleashed
 Co-
Co-Author of ADO.NET in 24 hours
 Author MS SQL Server High Availability
 Author Sybase Performance & Tuning
 Author Sybase Physical DB Design
 Author Chen Entity/Relationship Modeling
 Veritas SQL Server Performance Series
pbertucci@dbarchitechs.com

53. To learn more or inquire about speaking opportunities, please contact:
Mark Ginnebaugh, User Group Leader
Mark Ginnebaugh User Group Leader
mark@designmind.com