The document discusses Microsoft SQL Server data warehousing solutions. It provides an agenda for a presentation that includes an overview of Microsoft's data warehousing offerings, how to establish baseline metrics for Fast Track reference configurations, and how to design balanced server and storage configurations for data warehousing workloads. It also discusses software and hardware best practices, such as data striping and storage configuration recommendations. Overall, the document outlines topics and solutions to help customers accelerate their data warehouse deployments using Microsoft SQL Server.

1. Microsoft SQL Server Data
Warehouses for SQL DBAs
SQL Saturday Philly June 9, 2012

2. http://mssqldude.wordpress.com
http://www.sqlmag.com/blog/sql-server-bi-blog-17
mkromer@microsoft.com
http://joedantoni.wordpress.com
jdanton1@yahoo.com

3. Agenda
•
•
−
−
•
−
−
•
−
•
•
−
−

4. Microsoft Data Warehousing
Offerings
Tier 1 Offerings
Fast Track Data HP Business DW Parallel Data
Enterprise
Warehouse Appliance Warehouse
Appliance for high end Data
Scalable and reliable platform Reference Architectures offering An affordable SMP solution for
Warehousing requiring highest
for Data Warehousing on any best price performance for Data data warehousing on optimized
scalability, performance or
hardware Warehousing hardware
complexity
Ideal for data marts or small to Ideal for data marts or small to
Ideal for small data marts or DWs Offers flexibility in hardware and
mid-sized enterprise data mid-sized DWs with scan centric
with scan centric workloads architecture
warehouses (EDWs) workloads
DW Appliance
Reference Architectures Integrated Appliance
Software only (Fully integrated Software and
(Software and Hardware) (Software and Hardware)
Hardware)
Scale out data warehousing
Scale up data warehousing Scale up data warehousing Scale up data warehousing with massively parallel
processing (MPP)
10s of terabytes 4–80 terabytes Up to 5 terabytes 10s–100s of terabytes

5. Some Data Warehouses today
Big SAN
Big SMP Server
Connected together
What’s wrong with this picture?

6. Answer: system out of balance
 This server can consume 12 GB/Sec of IO, but the
SAN can only deliver 2 GB/Sec
 Even when the SAN is dedicated to the SQL Data
Warehouse, which it often isn’t
 Queries are slow
 Despite significant investment in both Server and Storage
Result: significant investment, not delivering performance

8. The Alternative: A Balanced System
 Design a server + storage configuration that can
deliver all the IO bandwidth that CPUs can
consume when executing a SQL Relational DW
workload
 Avoid sharing storage devices among servers
 Avoid overinvesting in disk drives

10. SQL Server Fast Track Data Warehouse
Solution to help customers and partners
accelerate their data warehouse deployments
 A method for designing a cost-effective,
balanced system for Data Warehouse
workloads
 Reference hardware configurations
developed in conjunction with hardware
partners using this method
 Best practices for data layout, loading and
management

11. Software:
• SQL Server 2008 R2
Enterprise
• Windows Server 2008 R2
Configuration guidelines:
• Physical table structures
• Indexes
• Compression
• SQL Server settings
• Windows Server settings
• Loading
Hardware:
• Tight specifications for servers,
storage and networking
• ‘Per core’ building block

12. Core Fast Track Metrics
•
−
−
−
−

13. System Benchmarking - MCR
•
−
−
•
−
•
− 200MB/s per core

14. Establishing Fast Track MCR
•
−
−
•
−

15. System Benchmarking - BCR
•
−
−
• Actual Miles Per Gallon
•

16. Establishing Fast Track BCR
•
−
−
−
−

18. Fast Track Reference Configurations
2 Processor Configurations (5 – 20 TB, 2-3.7 GB/s)




4 Processor Configurations (20 – 40 TB, 3.5-7.5 GB/s)




8 processor Configurations (40 – 80 TB, 7.5-14 GB/s)


19. Data Warehouse Workload Characteristics
SELECT L_RETURNFLAG, L_LINESTATUS, SUM(L_QUANTITY) AS SUM_QTY,
SUM(L_EXTENDEDPRICE) AS SUM_BASE_PRICE,
SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)) AS SUM_DISC_PRICE,
SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX))
AS SUM_CHARGE,
AVG(L_QUANTITY) AS AVG_QTY,
AVG(L_EXTENDEDPRICE) AS AVG_PRICE,
AVG(L_DISCOUNT) AS AVG_DISC,
COUNT(*) AS COUNT_ORDER
FROM LINEITEM
GROUP BY L_RETURNFLAG,
L_LINESTATUS
ORDER BY L_RETURNFLAG,
L_LINESTATUS

20. Software configuration
SQL Server Startup
•
−
•

21. Software configuration
Temp DB
•
−
−
•
−
•
•
−
−

22. Software configuration
Temp DB & TLOG
•
−
−
−
−
•
−
−
−
•
−
−

23. DW Server Baseline Configs
•
−
−
−
−
−
•
−

25. Fast Track Data Striping
•
FT Storage Enclosure
Raid-1
Primary Data Log
ARY01D1v01 ARY02D1v03 ARY03D1v05 ARY04D1v07 ARY05v09
DB1-1.ndf DB1-5.ndf DB1-7.ndf DB1.ldf
DB1-3.ndf
Disk 1 & 2 ARY01D2v02 ARY02D2v04 ARY03D2v06 ARY04D2v08
DB1-2.ndf DB1-4.ndf DB1-6.ndf DB1-8.ndf
Microsoft Confidential

26. User Databases
•
−
−
−
•
•
•
−

27. Transaction Log
•
•
•

28. LUN 1 LUN 2 LUN 3 LUN16
Permanent FG
Permanant_DB
Permanent_1.ndf Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf
Stage FG
Database
Stage
Stage_1.ndf Stage_2.ndf Stage_3.ndf Stage_16.ndf
Local Drive 1
TempDB
TempDB.mdf (25GB) TempDB_02.ndf (25GB) TempDB_03ndf (25GB) TempDB_16.ndf (25GB)
Log LUN 1
Permanent DB
Log
Stage DB Log

30. Control rack Data racks
Control Rack Data Rack
Compute Nodes Storage Nodes
Control Nodes SQL
Active / Passive
SQL
SQL SQL
SQL
Management Nodes
Dual Fiber Channel
SQL
Dual Infiniband
SQL
SQL
Landing Node
SQL
SQL
Backup Node
SQL
Spare Compute Node
Private Network

31. 1 Data Rack
• 17 Servers
• 22 Procs
• 132 Cores
Control Rack DataRack
Expand to 4 data racks and
quadruple your performance and capacity!

32. Query Speed in Seconds
PDW Time Orig. Time
4500 4200
4000
3500
3000
2500
2000
1500 1200 1200
1000
500 16 6 2 120 2 120 2 120 4
0
Q1 Q2 Q3 Q4 Q5 Q6
263x 200x 60x 60x 60x 300x
PDW times faster than original query speeds

33. Parallel Data Warehouse Appliance
Hardware Architecture
Compute Nodes Storage Nodes
Control Nodes SQL
Active/Passive
SQL
SQL
Client Drivers SQL
SQL
Management Nodes SQL
Dual Fiber Channel
Data Center Dual Infiniband SQL
Monitoring
SQL
Landing Node SQL
ETL Load Interface
SQL
Backup Node SQL
Corporate Backup
Solution
Spare Compute Node
Corporate Network Private Network

34. Parallel Data Warehouse benefits
Massively Parallel Processing
Compute Nodes Storage Nodes
Control Nodes ? SQL
Active/Passive Query 1 is
Query 1
? SQL
submitted to
SQL Server
SQL ? SQL
on Control
Node
? SQL
Management Nodes ? SQL
Dual Fiber Channel
Query is
Dual Infiniband ? SQL
executed on
all 10 Nodes
? SQL
Landing Node
? SQL Results are
sent back to
? SQL client
Backup Node ? SQL
Spare Compute Node
Corporate Network Private Network

35. Parallel Data Warehouse benefits
Massively Parallel Processing
Compute Nodes Storage Nodes
Control Nodes Multiple
???????? SQL
queries are
? Active/Passive
???????? SQL
simultane-
? ???? SQL
???????? SQL
ously
???
executed
? ? across all
???????? SQL
nodes.
? Management Nodes ???????? SQL
Dual Fiber Channel
Dual Infiniband ???????? SQL
? ???????? SQL
PDW
supports
? Landing Node ???????? SQL querying
???????? while
SQL data is
? ???????? loading.
? Backup Node SQL
Spare Compute Node
Blazing fast performance by parallelizing queries on highly optimized
Corporate Network Private Network
shared nothing nodes

36. •
•
•
−
−

37. MPP Engine Coordinator
Software Architecture Provides single system image
SQL compilation
Global metadata and appliance configuration
Global query optimization and plan generation
Global query execution coordination
Other Global transaction coordination
Query MS BI Internet Authentication and authorization
DWSQL Third- Explorer
Tool (AS, RS) Supportability (hardware and software status)
Party Tools
Compute Node
Compute Nodes
Compute Nodes
IIS Data Movement Service
Data Access Admin
(OLEDB, ODBC, ADO.NET, JDBC)
Console
User Data
SQL Server
Core
SQL DMS
Engine
Parser Manager Data Backup Node
Services
Movement
MPP Engine Coordinator Service Data Movement Service
Landing Zone Node
DW DW DW Data Movement Service
TempDB
Authentication Configuration Schema
SQL Server
Data Movement Service
Control Node Data movement across the appliance
Distributed query execution operators

39. Blazing-Fast Performance
“400 percent
improvement in
performance
First American Title
Insurance Company
Now, up to 10xFaster³
ColumnStore
¹Source: Microsoft customer evidence, Choice Hotels International
²Source: Microsoft customer evidence, KAS Bank
³Source: Microsoft customer testing; common data warehousing queries

40. ProductKey SalesAmount
OrderDateKey
OrderDateKey ProductKey SalesAmount
20101107 106 30.00
20101107
StoreKey RegionKey Quantity 103
17.00
20101107
01 1 6 109
20101107 20.00
2 1 103
04 20101107
17.00
2 2 106
04 20101108
2 20.00
1 106
03
3 OrderDateKey 25.00
4
05 1 20101108 ProductKey
5
02 20101108 SalesAmount
102
RegionKey
Quantity 20101108
106 14.00
StoreKey 1
1 20101109
109 25.00
02 2
5 20101109
1
106 10.00
03 20101109
1 106
01 2 20.00
4 103
2
04 25.00
1
5
04 1
17.00
01

41. 41
• Batch object
•
Column vectors
•
List of qualifying rows
−
−
•

43. In a standard scale-out server deployment, multiple report servers share a single
report server database. The report server database should be installed on a
remote SQL Server instance. The following diagram is an example of a standard
scale-out server deployment configuration with the report server database on a
remote SQL Server instance.

44. As another option, you might decide to host the report server database on a
SQL Server instance that is part of a failover cluster. The following diagram is
an example of a scale-out server deployment configuration where the report
server databases are on an instance that is part of a failover cluster.

45. In addition to the standard scale-out deployment, you might determine that your reporting environment
would benefit from a more advanced scale-out deployment configuration. For example, you might decide
to use the load-balanced report servers for interactive report processing and add a separate report server
computer to process only scheduled reports. The following diagram is an example of this advanced scale-
out server deployment configuration.

46. Log Description
The report server execution log contains data about specific reports, including when a report was run,
Report Server Execution Log who ran it, where it was delivered, and which rendering format was used.
The execution log is stored in the report server database.
The service trace log contains very detailed information that is useful if you are debugging an
Report Server Service Trace Log application or investigating an issue or event. The file is located at Microsoft SQL Server<SQL Server
Instance>Reporting ServicesLogFiles.
The HTTP log file contains a record of all HTTP requests and responses handled by the Report Server
Web service and Report Manager. HTTP logging is not enabled by default. You must modify the
Report Server HTTP Log
ReportingServicesService.exe configuration file to use this feature in your installation. The file is
located at Microsoft SQL Server<SQL Server Instance>Reporting ServicesLogFiles.

48. •
−
•
•
•
•
•

49. •
−
−
−
−
−
•
•
•
•
•
•
•
•
•
•
•
•
•

50. •
•
•
•
•
•
•

51. Under the properties of your data source, increasing the network packet size for SQL
Server minimizes the protocol overhead require to build many, small packages. The
default value for SQL Server 2008 is 4096. With a data warehouse load, a packet size of
32K (in SQL Server, this means assigning the value 32767) can benefit processing. Don’t
change the value in SQL Server using sp_configure; instead override it in your data source.
This can be set whether you are using TCP/IP or Shared Memory.

53. •
•
•
•
•
•
•
•
•
•
•
•
•
•
•

54. •
−
•
−
−
−
•
•
•
•

58. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.