This document provides an overview of using PowerShell to automate administrative tasks in SQL Server. It discusses SQL Server PowerShell (SQLPS), which loads the necessary SQL Server snap-ins. It also covers using SQL Server Management Objects (SMO) and ADO.NET for administrative tasks like managing servers and running queries. Key objects in SMO and ADO.NET that are useful for automation are highlighted.
2. Who am I ?
• Married + 2.5 children
• A DBA for 10 years.
• Oracle And Sql-Server Certified Professional.
• Used to be CTO@johnbryce training.
• Lead my own consulting business.
• Advisor to Tapuz.co.il / BezeqInt / Lavie Time-tec and more
More about me @ Www.dbconsultant.co.il
3. Topics
Introduction to PowerShell
Using PowerShell to automate administrative tasks
Governing Your Enterprise with PolicyBased Management
Extend Policy Based Management to the Enterprise Scaling PowerShell
Evaluation
Enhance your usage of DMVs
Developing maintenance tasks using SSIS Multi Server Administration
in SQL Server 2008 R2
Deploying apps using the Data Tier
5. Powershell
New Command-line shell & Scripting Language
Improves productivity & control
Accelerates automation of system admin
Easy-to-use
Works with existing scripts
6. More
Control
TechNet ScriptCenter
Exchange Server 2007
Terminal Server
Hundreds of Scripts WMI, Registry, Hardware, etc.
Community-Submitted scripts
MyITForum.com
Books & Training Manning Publications
O’Reilly Media
Materials Sapien Press & others…
MS MVPs
PowerShell Team Blog
Community Support Active Newsgroup
Channel 9: DFO Show
IIS.net
7. What is Powershell?
Revolutionary new interactive shell and scripting language
•Based on .NET
•New set of built-in tools (+120)
•New language to take advantage of .NET
•A new “object-pipeline” system view
•Can continue to use current tools
•Can continue to use current automation (COM)
8. Benefits
•Automate administration of multiple servers through a task-oriented scripting
language
•Accelerate script authoring, testing and debugging and write customer tools in a
new command shell environment
•Utilize new scripts and Cmdlets
•Manage command-line services, processes, registry, and WMI data
•Manage and/or automate administration tasks for server roles such as IIS and
Active Directory
•Automate Terminal Server configuration changes by means of PowerShell scripts,
and examine configuration similarities and differences across a Terminal Server
farm.
•Manage an Internet Information Services 7.0 environment.
•Remotely manage servers.
9. No Fear, Uncertainty or Doubt
• Do I need to learn .NET before I can use PowerShell?
– No - you can continue to use existing tools.
• Do I need to rewrite all my existing tools?
– No - existing tools will run just fine.
• Do I need to learn the new language?
– No - You can easily run existing commands without modification.
• Learning the new
– Online help is full of examples that are ready to use
– The new language elements make interacting with .NET a snap.
– Using .NET can help you where new tools don’t exist and opens a vast space
of productivity
– Learn at your own pace, PowerShell will be ready for you
10. 10 reasons why you should learn to use
PowerShell
1: It’s not going away any time soon
Microsoft has made it clear that PowerShell is here to stay. In fact,
PowerShell version 2 is not only included in Windows Server 2008 R2
and in Windows 7, it is enabled by default. Part of the reason why
Microsoft has done this is that going forward, many add-on products
will be based on PowerShell.
2: Most Microsoft products will eventually use it
Virtually all of the server products Microsoft is producing right now can
be managed through PowerShell. From an administrative standpoint,
this means that if you become proficient in PowerShell, you will have
the skill set necessary for managing most of Microsoft‘s newer
products. The basic built-in PowerShell commands are used in every
product that supports PowerShell. However, some server products
extend PowerShell to include additional cmdlets.
11. 3: You can’t do everything from the GUI any more
When Microsoft created Exchange 2007, it designed the GUI so that it
could be used only for the most common administrative functions.
Any obscure functions or anything potentially destructive has to be
performed using PowerShell. I expect this design philosophy to carry
over to other Microsoft products.
4: It can make your life easier
Believe it or not, using the command line can make your life easier.
Suppose for a moment that you need to update an Active Directory
attribute for a thousand users. Performing the task manually would
likely take hours to complete. Using PowerShell, though, you can
complete the task using a single line of code.
12. 5: Many GUIs are PowerShell front ends
Many of the GUI interfaces that Microsoft has been designing for its
various products are actually front end interfaces to PowerShell.
Probably the best known example of this is the Exchange
Management Console. Although this utility looks like a standard
management tool, it is built entirely on top of PowerShell. Any
function you perform through the GUI actually generates PowerShell
code that completes the requested task. In many cases, the console
even shows you the PowerShell command that was used at the
completion of the task.
6: Microsoft certification exams contain PowerShell questions
Microsoft has been adding PowerShell-specific questions to many of its
new certification exams. My experience with these exams has been
that you don‘t necessarily have to know the full command syntax, but
you do need to know which command you should be using in a
given situation.
13. 7: You can use PowerShell commands to manage your domains
If you have domain controllers running Windows Server 2003 with
Service Pack 2 or higher, you can install the Active Directory Web
Services on at least one domain controller. After doing so, you will be
able to use the Windows 7 RSAT Suite to manage Windows 2003 and
Windows 2008 domains.
8: It enables interactivity between products
PowerShell is the common thread between all the new server products
Microsoft is creating, so I expect to start seeing PowerShell used as a
mechanism for providing interactivity between server products. I
have yet to see a real world example of this interactivity, but
eventually I would expect to be able to use a PowerShell script to
work seamlessly between products such as IIS, SQL Server, and
Exchange.
14. 9: Microsoft says it’s important
Just because someone at Microsoft says that something is important,
that doesn‘t mean I take it as gospel. However, In the October 2009
issue of TechNet Magazine, Microsoft says, ―It‘s safe to say that the
single most important skill a Windows administrator will need in the
coming years is proficiency with Windows PowerShell.‖
Such a bold statement is hard to ignore. This is especially true given the
fact that this statement mirrors what I‘ve been hearing from various
people at Microsoft every time I have made a trip to Redmond lately.
10: If you don’t learn it, someone else will
As we all know, the economy is in a slump, and many companies are
downsizing. Needless to say, there is a lot of competition for the few
IT jobs that are available. Therefore, if you suddenly find yourself
looking for another job, your odds of finding one may be better if
you can list PowerShell among your skill set.
15. CMDlets
•A cmdlet (pronounced "command-let")
•is a single-feature command that manipulates objects in Windows
PowerShell.
•You can recognize CMDlets by their name format –
•a verb and noun separated by a dash (-),
•Get-Help
•Get-Process
•Start-Service.
16. Powershell - Recommendations
Start using Windows PowerShell
immediately!
Don‘t throw away any existing scripts or
batch files – they can still be used!
Don‘t forget the power of the wildcard, such
as ―*‖
Don‘t deploy Windows PowerShell on any
machine where it is not actually needed
Centrally-Control Windows PowerShell
security settings through GPOs – do it now!
18. PowerShell: variable syntax
Basic syntax similar to all scripting languages
$a = ―value‖
$a = 1 + 2
$a += 137
Built-in support for types
$a = [xml]―<test><a>avalue</a></test>‖
[int]$a = 42
or cast to arbitrary .NET object
19. PowerShell: parsing
Two contexts:
command context
starts with regular character
tries to execute command
expression context
starts with num, var, or quoted string
executes as expression
Can reset mode with ―( )‖
(Get-Date).day + 2
20. PowerShell: array and hash
$a = 1,2,3,4
$a += 5..10
+ adds elements to array
.. is range operator. What is $a[1..3]?
$a[1], $a[2], $a[3]
$a = @{ one=1; val = get-childitem }
$a[‗val‘]
$a.one
22. PowerShell: looping
while($a –lt 137) { <block> }
foreach($var in get-process) { <block> }
IEnumerable support
Shorten foreach to %
ls | %{ $_.Length }
receive piped objects
$_ as before
for { $i = 0; $i –lt 10; $i += 2} { $i }
regular for loop
23. PowerShell: useful operators
&, .
call operator
$a = ―Get-childitem‖; &$a # calls get-childitem
. used for executing scripts in current context
-as, -is
is/as in C#
$a –as [int]
if { $a –is [System.DateTime] } { … }
24. PowerShell: functions
function [(args)] { <body> }
if Param is first statement, then gives parameters
arguments passed in $args
input passed in $input
parameter passing on cmd line
add –x 2 –y 3
add 2 3
25. Useful commands
get-member
return the members of an object
eg. get-member –MemberType property
or, method
Authenticode signing
can use certificates to verify scripts
checks the hash and returns
26. PowerShell: errors
throw
throw ―error‖
same as ThrowTerminatingError in Cmdlets
trap
catch exceptions
trap [DivideByZeroException] { <do something> }
break/continue semantics
27. PowerShell: surprises
Drives
C, D
Env, Alias, HKLM, variable, function
mount lets you create others
variable scoping
private, local, script, global
30. Scriptblocks
a chunk of script
a type in PowerShell
$x = [scriptblock]{$y = 137}
&$x;
Can be used as an EventHandler delegate
this is what‘s happening in the last example
object parameter -> $this
EventArgs parameter -> $_
31. Cmdlet creation
Subclass of System.Management. Automation.Cmdlet
must have an attribute
gives ―verb‖ and ―noun‖ components of name
overrides at least one of
BeginProcessing, ProcessRecord, EndProcessing
use make-shell command to include it
adds dll to set of cmdlets
requires some registry manipulation
32. Cmdlet creation
Cmdlet parameters
add [Parameter] attribute
Mandatory=true|false
Position=<index>
… others for in-pipeline action
Return value: any object
native object facilities allow inspection
can use arbitrary code
33. Example in script
function MyWhere {
param ( [scriptblock]$expression )
begin { $matches = 0 }
process {
if ( &$expression ) { $_; $matches++ }
}
end { "Found $matches matches" }
}
34. Execution Policy
AllSigned
require a digital signature and prompt
user must agree to run script
RemoteSigned
only files from internet need to be signed
default setting
Unrestricted
no signing required
Attacks on AllSigned?
35. Using PowerShell to Manage And automate
administrative tasks
SQLPS.exe – The SQL Server Mini-Shell
SQL Server Management Objects – SMO
Using ADO.NET for Queries
36. To make things easier for administrators using PowerShell for SQL
Server, Microsoft created an executable - a mini-shell - which
automatically loads the SQL Server PowerShell snap-ins. The
program is called sqlps.exe and is included when you install SQL
Server 2008. Besides automatically loading the snap-ins the
sqlps.exe environment enables script execution automatically by
setting the execution policy to RemoteSigned.
37. This mini-shell can be invoked in a number of ways. You can use the
Windows Start, Run menu item and type in sqlps and start it that
way. SQL Server Agent jobs can use the SQL Server Agent
PowerShell subsystem in a step to run a script by selecting the step
type of PowerShell. Each time a step runs the PowerShell step it
loads a separate copy of sqlps.exe into memory, each taking about
40MB, so caution is advised when using this feature.
38. The most interesting method is within SQL Server Management
Studio. In Object Explorer you can right-click on any object in the
―tree‖ under an instance and select ―Start PowerShell‖, and sqlps.exe
will start up, and the Set-Location cmdlet is used to set the current
location in the shell window to the object you pointed to. This
provides a very nice ability to do some ad-hoc browsing of SQL
Server objects as necessary.
39. SQL Server Management Objects - SMO
SQL Server Management Objects (SMO) is the object library
provided by Microsoft to perform administrative actions against SQL
Server. The library is built on top of the .NET Framework (2.0) so
applications written using SMO are using managed code and benefit
from the features available within the Framework, including the
security and cleanup features. SMO was introduced with SQL Server
2005 and allows you to manage servers running SQL Server 2000,
SQL Server 2005, and SQL Server 2008.
40. Using ADO.NET for Queries
ADO.NET has two sets of objects, a connected set that allows you to
connect with a data source, run queries and return result sets, and a
disconnected set of objects that allow you to work with the data
after it‘s been collected. Here‘s a list of the objects.
ADO.NET Object Description
Connection
Object A connection to the data source
Can represent a query against a database, a call to a stored procedure, or a direct
Command Object request to return the contents of a specific table
DataReader
Object Designed to return query results as quickly as possible
Transaction
Object Groups a number of changes to a database and treats them as a single unit of work
The Connection object has a BeginTransaction method that can be used to create
Transaction objects
Parameter Object Allows the specification of parameters for stored procedures or parameterized queries
DataAdapter Acts as a bridge between the database and the disconnected objects in the ADO.NET
Object object model
41. ADO.NET Object Description
Allows the examination of data through collections of rows and
DataTable Object columns
DataColumn
Object Corresponds to a column in a table
Constraint Object Defines and enforces column constraints
DataRow Object Provides access to the DataTable's Rows collection
DataSet Object The container for a number of DataTable objects
DataRelation
Object Defines the relations between DataTables in the DataSet object
DataView Object Allows the examination of DataTable data in different ways
42. Using the Set-ExecutionPolicy Cmdlet
The Set-ExecutionPolicy cmdlet enables you to determine which
Windows PowerShell scripts (if any) will be allowed to run on your
computer. Windows PowerShell has four different execution policies:
Restricted - No scripts can be run. Windows PowerShell can be
used only in interactive mode.
AllSigned - Only scripts signed by a trusted publisher can be
run.
RemoteSigned - Downloaded scripts must be signed by a
trusted publisher before they can be run.
Unrestricted - No restrictions; all Windows PowerShell scripts
can be run.
44. Conclusion
Scripting has been a powerful tool for Unix administrators for a long
time. Windows administrators have had fewer and less capable
options for automating administrative processes until the
introduction of PowerShell.
There are a great many sites providing quality information on using
PowerShell to automate administrative tasks. The addition of
PowerShell support to SQL Server 2008 tool set adds an extra
dimension of manageability to the Windows Server
environment. You should review the available books and online
material to develop your skills in using this powerful tool, and by
doing so automate and streamline the processes in your own
environment.
46. Enterprise Policy Management
• Provide auditors with assurance that SQL Server
Compliance complies with all security and business guidelines
• Complement All Actions Audited
Consistency
• Ensure peak performance
• High levels of security & reliability
• Drive strategic management initiative to control
Costs costs
• More efficient and proactive management
47. Policy-Based Management Framework
Introduction to the Framework
Evaluation Modes
Extending Policy-Based Management to the Enterprise
Solution Architecture
Considerations for Large-Scale Environments
Centralized Reporting
48. Policy-Based Management
Defines the evaluation mode, target filters, and schedule of the conditions.
Policy
Specifies a set of allowed states of a managed target with regard to a facet
Condition
Set of related logical properties
Facet
49. Server
Restriction
Category Policy Target
Evaluation
Mode
50. Policy Evaluation Modes
On Demand On Schedule
• Evaluate a policy when specified by user • SQL Server 2008 only
• Available through SSMS or Windows • SQL Server Agent job periodically evaluates
PowerShell™ a policy
• Option to force certain conditions to
comply with policy
• Supports down-level evaluation
(depends on properties exposed)
Evaluation modes
On Change: Prevent On Change: Log Only
• SQL Server 2008 only • SQL Server 2008 only
• DDL triggers prevent policy violations • Event notification evaluates a policy when a
relevant change is made
51. On Demand Evaluation
Windows PowerShell™ Integration
Windows PowerShell™ is a framework and runtime for executing
management commands
Cmdlets are instances of .NET classes that process input objects from the
pipeline
SQL Server Provider for Windows PowerShell™ encompasses SMO
Invoke-PolicyEvaluation –Policy
DatabaseStatus.xml,
Trustworthy.xml -TargetServerName inst1
Invoke-SQLCMD –Query ”SELECT name FROM
sys.Databases;” –ServerInstance
“MyServerInstance”
52. Extend you enterprise- Bringing it all together
policy
results policy
results
policy
results policy
results
policy
results policy
results
54. Extend to the Enterprise
Bringing It All Together
policy
results
policy
results
policy
results policy
results
policy
results
policy
results
55. Extend to the Enterprise- Central Management
Server
Logically group instances based on
business function(s)
Centrally publish policies to groups of
SQL Server 2008 instances
Evaluate policies on-demand against
a group of servers
Filter by logical groups in Windows
PowerShell™ scripts
56. Extend to the Enterprise- Scaling PowerShell
Evaluation
Add Intelligence to Policies
Place each policy in a category
Define server restrictions for
versions and editions where
appropriate
57. Extend to the Enterprise- Scaling PowerShell
Evaluation
Create Custom Server Groups in the
CMS
Run specific policies against a list
of servers
Examples: Production,
Development, PCI
Define Concurrent Jobs
Define multiple concurrent
executions based on Policy
Category and/or logical Central
Management Server group
58. Extend to the Enterprise
Benefits of Upgrading to SQL Server 2008
Real-Time Enforcement and
Reporting
Monitor the event log through
Alerting integration
Advanced functionality and
integration with SSMS
Dependency, health states,
subscriptions, history
Scale
Security
Access to other rich features in SQL
Server 2008
59. Centralizing Policy History
SQL Server 2008
policy
results
policy
results
policy
results
syspolicy_policy_execution_history policy
results
syspolicy_policy_execution_history_details
60. Additional Resources
Policy Based Management
Enterprise Policy Management Framework
http://www.codeplex.com/EPMFramework
Policy Based Management Blog
http://blogs.msdn.com/sqlpbm/default.aspx
Lara Rubbelke‘s Blog
http://sqlblog.com/blogs/lara_rubbelke/default.aspx
Dan Jones‘ Blog
http://blogs.msdn.com/dtjones/default.aspx
Buck Woody‘s Blog
http://blogs.msdn.com/buckwoody/default.aspx
61. Additional Resources
Windows PowerShell™
To learn more about the Windows PowerShell™ scripting Language
http://www.microsoft.com/downloads/details.aspx?FamilyID=b4720b
00-9a66-430f-bd56-ec48bfca154f&DisplayLang=en
Windows PowerShell™ Blog
http://blogs.msdn.com/powershell/
Allen White‘s Blog
http://sqlblog.com/blogs/allen_white/
SQL Server PowerShell Overview
http://msdn.microsoft.com/en-us/library/cc281954.aspx
62. Extend Policy Based Management to the
Enterprise Scaling PowerShell Evaluation
The Enterprise Policy Management Framework (EPM) is a solution to
extend SQL Server 2008 Policy-Based Management to all versions of
SQL Server in an enterprise, including SQL Server 2000 and SQL
Server 2005. The EPM Framework will report the state of specified
SQL Server instances against policies that define intent, desired
configuration, and deployment standards.
63. When the Enterprise Policy Management Framework (EPM) is
implemented, policies will be evaluated against specified instances of
SQL Server through PowerShell. This solution will require at least one
instance of SQL Server 2008. The PowerShell script will run from this
instance through a SQL Server Agent job or manually through the
PowerShell interface. The PowerShell script will capture the policy
evaluation output and insert the output to a SQL Server table. SQL
Server 2008 Reporting Services reports will deliver information from
the centralized table.
64. This solution requires the following components to be configured in
your environment. All SQL Server 2008 requirements listed below
may be executed from and managed on the same instance:
§ SQL Server 2008 instance to store policies
§ SQL Server 2008 instance to act as the Central Management Server
§ SQL Server 2008 instance to execute the PowerShell script
§ SQL Server management database to archive policy evaluation results
§ SQL Server 2008 Reporting Services to render and deliver policy
history reports
65. Extend Policy Based Management to the
Enterprise Scaling PowerShell Evaluation
67. Overview
SQL Server 2000 provided
~11 PerfMon objects and counters
Access to some system tables
A few table-valued functions
SQL Server 2005 introduced numerous long-awaited performance
measurement metrics
Compared to SQL 2000, SQL 2005
Approximately tripled SQL-specific PerfMon objects and counters
Created numerous views and functions (~80 Dynamic Management Views
and Functions) that provide very useful and previously unavailable
performance information
SQL 2008
Added four SQL-specific PerfMon objects and counters
Added ~40 Dynamic Management Views and Functions
67
68. Overview
Sheer number of performance views and new metrics can be dizzying
New metrics documented in
Books Online
Numerous articles and books
However, much of the literature simply defines the DMVs and metrics,
so knowing where to begin is extremely difficult
http://www.microsoft.com/downloads/details.aspx?FamilyID=531c53e7
-8a2a-4375-8f2f-5d799aa67b5c&displaylang=en
68
69. Overview
Performance analyst must understand
Which PerfMon objects and DMVs are pertinent to a particular
problem
How to interpret numerous metrics provided by new objects and
views
Most valuable views and metrics
69
70. Emphasis of Presentation
Highlight most useful of the newly available
SQL Server-specific PerfMon objects and counters
DMVs
How to convert the numeric DMV identifiers into understandable text
Properly use and interpret most important
SQL Server-specific PerfMon objects and counters
DMV metrics
70
71. SQL Server PerfMon Objects
Many more useful metrics, e.g.,
Wait statistics
Tempdb (General Statistics)
Cursors
Transactions
.Net Data Provider for SQL Server
Deprecated Features (2008)
71
72. Wait Statistics Object
Subset of this information was previously available only via complex
queries from several disparate sources
Small overlap with Locks object
Instances where Counters would normally be
Most useful instances
Cumulative wait time (ms)
Average wait time (ms)
72
73. Wait Statistics Object
Cumulative wait time (ms) instance extremely useful
Unfortunately, still only counts values within the last second
Sampling every 15-30 seconds usually provides an adequate
representation
Use sys.dm_os_wait_stats to determine
Total wait times by specific type of wait without sampling problems
Discussed later in presentation
73
75. Lockable PerfMon Resources
Resource Description
AllocUnit (2005/2008) Allocation unit
Application Application-specified resource
(2005/2008)
Database Entire database
Extent Contiguous group of 8 data or index pages
File (2005/2008) Database file
HoBT (2005/2008) Heap or B-Tree index
Key Row lock within an index that protects range of keys in
serializable transactions
Metadata (2005/2008) Meta data or catalog information
Object (2005/2008) Job flow synchronization object, table, stored procedure, or
view
Page 8-kilobyte (KB) data or index page
RID Row ID. Used to lock a single row within a table
Table (replaced by Entire table, including all data and indices
Object on 2005/2008) 75
76. Deprecated Features Object
Simplifies detection of deprecated feature usage since last SQL Server
instance restart
Examples of deprecated features/counters
'::' function calling syntax
Data types: text ntext or image
DATABASEPROPERTY
DATABASEPROPERTYEX('IsFullTextEnabled')
DBCC DBREINDEX
DBCC INDEXDEFRAG
DBCC SHOWCONTIG
NOLOCK or READUNCOMMITTED in UPDATE or DELETE
sp_adduser
sysindexes
sysobjects
Use Deprecation Announcement & Deprecation Final Support trace
events to determine exact causes 76
77. Critical General Views
Required for converting numeric DMV identifiers into understandable text
Many were available under 2000 with slightly different names
2000 sysobjects -> 2005 sys.objects
See ―Mapping System Tables to System Views (Transact-SQL)‖ topic
in BOL for further info
sys.databases
Lists all databases and their IDs so proper associations can be made
sys.partitions
Only way to decode HOBT IDs returned by lock-specific information,
e.g. blocked process records
sys.configurations
Provides information regarding OS and SQL Server configurations
sys.dm_os_sys_info
Provides information regarding system (can be sampled once or
repeatedly)
Useful for determining whether hyperthreading is active and accessible77
78. sys.dm_os_sys_info Useful Columns
No parameters required
Number of logical CPUs on the system
Ratio of number of logical or physical cores exposed by one physical
processor package (hyperthread_ratio)
Amount of physical memory available
Amount of virtual memory available to the process in user mode.
This can be used to determine whether SQL Server was started by
using a 3-GB switch
Date and time SQL Server last started
78
79. Database-Specific Views
Query for each database separately
sys.objects
Lists all database objects such as tables, views, stored procedures,
etc.
sys.indexes
Lists all indices and their associated table IDs
Does not provide row counts as sysindexes does
sys.filegroups
Lists all file groups and their IDs
sys.database_files
Lists all physical database files and their IDs
79
80. Dynamic Management Views and Functions
“Designed to give you a window into what's going on inside SQL
Server”
Two types
DMV - Pure view, i.e., no parameters required
DMF - Table-valued function, i.e., parameters required
Parameters usually specify database, table, index, partition, etc.
Provide significant amount of information regarding
System
Databases
Internal workings of SQL Server
Performance
80
81. Dynamic Management Views and Functions
Some were possible with complex queries on 2000
When possible
Very clumsy to use
Required temp tables
Required significant understanding of the underlying tables
Most DMVs are very simple to use
Some still require joins with other DMVs and DMFs
Some may lessen need for using SQL Trace
81
82. Dynamic Management Views and Functions
Contain values since last SQL Server instance restart, unless manually
reset
Perfect for periodic or intermittent sampling
Therefore, must compute differences between adjacent or distant
samples for same database, file, table, index, etc.
Many DMVs can be reset manually using command similar to
DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR)
82
83. Dynamic Management Views and Functions –
General Guidelines
Most performance DMVs begin with sys.dm_
Must reconcile numeric IDs with static views that contain textual
names
Values accumulated from last SQL Server instance restart
Sample rates can range from once per minute to a few times per
day
Must calculate differences between individual sample records
Be sure to diff records with same database ID, object ID, and index
ID
83
84. Dynamic Management Views and Functions –
General Usage Scenario
Method 1
Capture before monitored activities begin and store results into a
SQL table or a spreadsheet
Execute workload (whether natural production or test)
Capture again and compare value differences
Load into spreadsheets for further analysis, if necessary
Method 2
Capture every <n> minutes and store results in a flat file
After workload and capture process complete, load data into SQL
tables
Use SQL to compare incremental value differences and store results
Extract interval data to spreadsheets for further analysis
84
85. Dynamic Management Views and Functions –
Usage Warnings
Microsoft assures us these views DO NOT BLOCK (single exception
cited below) as usage of older system tables could
However, be careful of execution frequency when cross apply or
several joins involved because resource consumption could be
high
Beware of the following two
sys.dm_db_index_physical_stats, which examines the data within
the physical files and provides extremely valuable data, can block
depending upon the option used, but not nearly as badly as
DBCC SHOWCONTIG
sys.dm_tran_locks can consume large amounts of CPU and can
generate huge output when # of locks large
Use only if absolutely necessary
85
86. Dynamic Management Views and Functions
Category List
View Category View Category
Change Data Capture Related Dynamic Query Notifications Related Dynamic
Management Views Management Views
Common Language Runtime Related Replication Related Dynamic Management
Dynamic Management Views Views
Database Mirroring Related Dynamic Resource Governor Dynamic Management
Management Views Views
Database Related Dynamic Management Service Broker Related Dynamic
Views Management Views
Execution Related Dynamic Management SQL Server Extended Events Dynamic
Views and Functions Management Views
Full-Text Search Related Dynamic SQL Server Operating System Related
Management Views Dynamic Management Views
Index Related Dynamic Management Transaction Related Dynamic Management
Views and Functions Views and Functions
I/O Related Dynamic Management Views Security Related Dynamic Management
and Functions Views
Object Related Dynamic Management
86
Views and Functions
87. Execution-Related Dynamic Management Views
and Functions
Most commonly used ones
sys.dm_exec_query_stats
sys.dm_exec_requests
sys.dm_exec_sessions
sys.dm_exec_sql_text
Other less commonly used ones
sys.dm_exec_cached_plans
sys.dm_exec_connections
sys.dm_exec_cursors
87
88. sys.dm_exec_query_stats DMV
Returns aggregate performance statistics for cached and completed
query plans
Top <n> can be returned for any category
Execution count, CPU time, CLR time, and elapsed time
Physical and logical reads, Logical writes
Activity Monitor on 2008 appears to use this DMV primarily
Currently active long-running queries NOT shown via this view
View contains one row per query plan
Lifetime of a row is tied to the plan itself
Can reduce dependence upon SQL statement tracing, but
creation_time, last_execution_time, and accumulated values MUST be
accounted for
88
89. sys.dm_exec_query_stats Columns
―Starting and ending positions of the query‖… ―within the text of its
batch or persisted object‖
Pointer to plan, which can be passed to the sys.dm_exec_sql_text
DMF
Times at which plan was compiled and last executed
# of times plan executed since last compilation
Total, minimum, maximum, and last amounts consumed by executions
since compilation
CPU, CLR, and elapsed times, all in microseconds
Physical and logical reads, Logical writes
89
90. sys.dm_exec_query_stats Example
Returns top 20 query statements that performed most logical reads
select top 20 [sql_handle], creation_time, execution_count,
last_execution_time, total_worker_time, total_physical_reads,
last_physical_reads, max_physical_reads, total_logical_reads,
last_logical_reads, max_logical_reads, total_logical_writes,
last_logical_writes, max_logical_writes, total_clr_time, last_clr_time,
max_clr_time, total_elapsed_time, last_elapsed_time,
max_elapsed_time,
SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(st.text)
ELSE qs.statement_end_offset END
- qs.statement_start_offset)/2) + 1) as statement_text
FROM sys.dm_exec_query_stats as qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as st
ORDER BY total_physical_reads DESC
Note how other DMVs and DMFs are used
90
91. sys.dm_exec_requests Columns
Timestamp when request arrived
Status of the request
Identifies current type of command being processed, e.g., select,
insert, update, etc.
Plan handles and offsets similar to sys.dm_exec_query_stats
If blocked, ID of session blocking request
If request currently (or previously) blocked, returns wait type and wait
time (ms)
# of transactions and result sets open for request
CPU, total elapsed times in (ms)
Physical and logical reads, Logical writes
# of rows that have been returned to the client by this request
91
92. sys.dm_exec_requests Example
Returns batches that contain top 20 currently active query statements
that consumed the most CPU
select top 20 *
from sys.dm_exec_requests ExecReq
OUTER APPLY sys.dm_exec_sql_text
(ExecReq.sql_handle) ExecSQLText
where (session_id > 50 and session_id <>
@@spid) and
command not like 'WAITFOR%'
order by cpu_time desc
92
93. Index Related Dynamic Management Views and
Functions
sys.dm_db_index_usage_stats
Returns information regarding query code usage of tables and
indices, e.g., inserts, updates, deletes, random accesses, and
sequential accesses
sys.dm_db_index_operational_stats
―Returns current low-level I/O, locking, latching, and access method
activity for each partition of a table or index in the database‖
Most commonly useful metric tracks actual forwarded record
fetches
sys.dm_db_index_physical_stats
Execute infrequently because physical files are interrogated and
causes some blocking to occur
Replaces DBCC SHOWCONTIG
93
94. sys.dm_db_index_usage_stats DMV
Returns
One row per index
Number of seeks, scans, lookups, and updates for user and system
queries
Seeks random
Scans sequential
Time of last seek, scan, lookup, and update for user and system
queries
System category
Maintenance, e.g., statistics updates
User category
Insert, update, delete, and select operations
Reports user statements, not record counts
One insert statement can result in millions of rows being inserted
View will report one, not millions
Helps determine index and table usage patterns
Particularly useful for identifying indices that are never used by 94
queries
95. sys.dm_db_index_usage_stats Example
List Rarely-Used Indices For Specific DB With Resolved Names
declare @dbid int = db_id()
select objectname=object_name(inxusage.object_id),
inxusage.object_id,
indexname=sysinx.name, sysinx.index_id, user_seeks, user_scans,
user_lookups, user_updates
from sys.dm_db_index_usage_stats inxusage,
sys.indexes sysinx
where database_id = @dbid and
objectproperty(inxusage.object_id,'IsUserTable') = 1 and
sysinx.object_id = inxusage.object_id and
sysinx.index_id = inxusage.index_id
order by (user_seeks + user_scans + user_lookups +
user_updates) asc
95
96. sys.dm_db_index_operational_ stats DMF
One row per index
Helpful for
Determining how indices are used
Identifying contention areas
Four parameters required
{ database_id | NULL | 0 | DEFAULT } (use db_id() for current db)
{ object_id | NULL | 0 | DEFAULT }
{ index_id | NULL | -1 | DEFAULT }
{ partition_number | NULL | 0 | DEFAULT }
Use NULL parameters to obtain information for all databases and files
select * from sys.dm_db_index_operational_stats (NULL, NULL,
NULL, NULL)
96
97. sys.dm_db_index_operational_ stats Columns Pt 1
Cumulative counts
Range and table scans started on index or heap
Single row retrievals from index or heap
Rows that were fetched through forwarding record
Row locks requested
Page locks requested
Cumulative counts and elapsed times database engine
Waited on row lock
Waited on page lock
Waited because of latch contention
Waited on I/O page latch
97
98. sys.dm_db_index_operational_ stats Columns Pt 2
Cumulative counts
Leaf-level insert, delete, update, delayed delete operations
Above leaf-level insert, delete, update, delayed delete operations
Zeroes heap operations
Leaf-level page allocations in the index or heap
For an index, page allocation corresponds to a page split
Page allocations caused by page splits above leaf level
Page compression attempt and success counts (2008)
98
99. sys.dm_db_index_operational_ stats Usage Pt 1
Reports actual operation counts, not user statements
One insert statement can result in millions of rows being inserted
View will report millions, not one
To analyze a common access pattern to the table or index partition
leaf_insert_count
leaf_delete_count
leaf_update_count
leaf_ghost_count
range_scan_count
singleton_lookup_count
99
100. sys.dm_db_index_operational_ stats Usage Pt 2
To identify latching and locking contention
page_latch_wait_count and page_latch_wait_in_ms
Indicate whether latch contention exists on index or heap, and
significance of contention
row_lock_count and page_lock_count
Indicate how many times the Database Engine tried to acquire
row and page locks
row_lock_wait_in_ms and page_lock_wait_in_ms
Indicate whether lock contention exists on index or heap, and
significance of contention
To analyze statistics of physical I/Os on an index or heap partition
page_io_latch_wait_count and page_io_latch_wait_in_ms
Indicate how many physical I/Os were issued to bring index or
heap pages into memory and how much waiting was involved
100
101. sys.dm_db_index_physical_stats DMF
Five parameters required
{ database_id| NULL | 0 | DEFAULT } (use db_id() for current db)
{ object_id| NULL | 0 | DEFAULT }
{ index_id| NULL | -1 | DEFAULT }
{ partition_number| NULL | 0 | DEFAULT }
{ LIMITED | SAMPLED | DETAILED | NULL | DEFAULT }
LIMITED mainly provides fragmentation information
Strict locking scheme not used for LIMITED or SAMPLED, so DETAILED
mode primary cause of any blocking
DETAILED provides much useful information
Record count, forwarded record count, min-max-avg record
lengths
101
102. sys.dm_db_index_physical_stats
Use NULL parameters to obtain information for all databases and files
select * from sys.dm_db_index_physical_stats (NULL, NULL, NULL,
NULL, 'DETAILED')
DETAILED mode
Used instead of SAMPLED when index or heap has < 10,000 pages
Provides information about non-leaf levels
102
103. sys.dm_db_index_physical_stats Columns
Index_type_desc
Index type
Index_id, index_level
0 heap
Index_depth
# of index levels
Avg_fragmentation_in_percent
Logical fragmentation for indexes or extent fragmentation for
heaps
Indices with values > 30 are candidates for rebuild/reorganization
Fragment_count
Number of fragments in the leaf level
Avg_fragment_size_in_pages
Average number of pages per fragment in the leaf level
Larger is better
Forwarded_record_count
# of heap records having forward pointers to another data location 103
104. I/O Related Dynamic Management Views and
Functions
sys.dm_io_pending_io_requests
sys.dm_io_virtual_file_stats
104
105. sys.dm_io_virtual_file_stats DMF
Returns I/O statistics for data and log files, one row per file
Use NULL parameters to obtain information for all databases and files
select * from sys.dm_io_virtual_file_stats (NULL, NULL)
Equivalent to select * from ::fn_virtualfilestats(-1,-1) on SQL 2000
105
106. sys.dm_io_virtual_file_stats Columns
ID of database
ID of file
Number of milliseconds since SQL Server instance started (useful for
detecting restarts)
Number of reads and writes issued against this file
Total number of bytes read from and written to this file
Total time, in milliseconds, users waited for I/O completions overall, as
well as reads and writes specifically
Number of disk bytes used by this file
106
107. SQL Server Operating System-Related Dynamic
Management Views
Abundant views
Most frequently used views
sys.dm_os_wait_stats
sys.dm_os_waiting_tasks
sys.dm_os_latch_stats
107
108. sys.dm_os_wait_stats DMV
Useful for diagnosing performance issues with
SQL Server
Specific queries and batches
Expands upon PerfMon Wait Statistics object
Returns information about waits encountered by threads in execution
226 possible wait types
Several of these are innocuous because they are accumulated as
part of normal SQL Server operation
Very important to eliminate normal wait types from
consideration
108
109. sys.dm_os_wait_stats
Very useful to categorize and group individual wait types to simplify
understanding and analysis, e.g.,
Combine LOGMGR, IMPPROV_IOWAIT, IO_AUDIT_MUTEX,
IO_COMPLETION, DISKIO_SUSPEND, IO_AUDIT_MUTEX,
IO_COMPLETION, ASYNC_DISKPOOL_LOCK,
ASYNC_IO_COMPLETION, REQUEST_DISPENSER_PAUSE,
PAGEIOLATCH_DT, PAGEIOLATCH_EX, PAGEIOLATCH_KP,
PAGEIOLATCH_NL, PAGEIOLATCH_SH, PAGEIOLATCH_UP into I/O
Combine PAGELATCH_DT, PAGELATCH_EX, PAGELATCH_KP,
PAGELATCH_NL, PAGELATCH_SH, PAGELATCH_UP into Latches
109
110. sys.dm_os_waiting_tasks DMV
Reports information about currently waiting tasks
Can be used to construct blocker/waiter lists
Important Fields
Total wait time (ms) for wait type
Name of wait type
Task currently holding this resource
Blocking task session ID
Description of resource being consumed
110
111. sys.dm_os_latch_stats DMV
SQL Server uses numerous kinds of latches (118 documented as of
2008 SP1)
Latch times can be helpful in verifying I/O subsystem performance
Large latch wait times often indicate poor I/O performance
View returns information regarding all latch types
Very useful to categorize and group individual wait types to
simplify understanding, e.g.,
BUFFER_POOL_GROW and BUFFER into Buffer
ALLOC_CACHE_MANAGER, ALLOC_CREATE_FREESPACE_CACHE,
ALLOC_CREATE_RINGBUF, ALLOC_EXTENT_CACHE, and
ALLOC_FREESPACE_CACHE into Allocation
111
112. Transaction Related Dynamic Management Views
and Functions
Numerous views
sys.dm_tran_locks is useful, but can distort system depending upon #
of locks being held
112
113. sys.dm_tran_locks DMV
Each row represents currently active request to lock manager for lock
that has been granted or is waiting to be granted
Result set columns are divided into two main groups — resource and
request
Resource group describes resource on which the lock request is
being made
Resource type same as lockable resource
Request group describes the lock request
113
114. Conclusions
SQL Server 2008 provides a wealth of performance information
Numerous Dynamic Views are essential for uncovering and resolving
performance bottlenecks
Dynamic Views and Functions are
Easy to use
Well documented
Low overhead with very few exceptions
114
115. New sys.dm_ DMVs and DMFs in SQL 2008
View Name View Name View Name
dm_audit_actions dm_fts_fdhosts dm_server_audit_status
dm_audit_class_type_map dm_fts_outstanding_batches dm_tran_commit_table
dm_cdc_errors dm_os_dispatcher_pools dm_xe_map_values
dm_cdc_log_scan_sessions dm_os_dispatchers dm_xe_object_columns
dm_cryptographic_provider_pro
perties dm_os_memory_brokers dm_xe_objects
dm_database_encryption_keys dm_os_memory_node_access_stats dm_xe_packages
dm_db_mirroring_auto_page_rep dm_xe_session_event_actio
air dm_os_memory_nodes ns
dm_db_mirroring_past_actions dm_os_nodes dm_xe_session_events
dm_xe_session_object_colu
dm_db_persisted_sku_features dm_os_process_memory mns
dm_db_script_level dm_os_spinlock_stats dm_xe_session_targets
dm_exec_procedure_stats dm_os_sys_memory dm_xe_sessions
dm_resource_governor_configurati
dm_exec_trigger_stats on
dm_resource_governor_resource_p
dm_filestream_file_io_handles ools
dm_resource_governor_workload_g 115
dm_filestream_file_io_requests roups
120. R2 makes it easier for DBAs to move databases around from server
to server in much the same way virtualization admins move guest
OS‘s around between physical hosts.
No more need for backup and restore.
Easily deploy databases.
121. The following steps illustrate a simple
deployment process:
A database developer completes the development of a Finance version
1.0 DAC, and then builds a FinanceVersion1.dacpac DAC package.
A database administrator deploys the Finance 1.0 DAC to production.
The production instance of the database engine now has a deployed
DAC that has an application name of Finance, a version of 1.0, and an
associated database named Finance.
The database developer then starts work on the next version, and when
development completes builds a FinanceVersion2.dacpac.
122. The database administrator then runs the Upgrade Data-tier Application
Wizard specifying both the current deployed version 1.0 DAC and the
version 2.0 DAC package. The wizard performs the following steps:
Verifies that the deployed DAC and the DAC package both have
their application names set to Finance. It also evaluates whether
the instance of the Database Engine meets the requirements
specified in the server selection policy (if defined) in the version
2.0 DAC.
Deploys the Finance version 2.0 DAC from the DAC package. This
creates a new database with a temporary name.
Sets the original database to read-only if it is not already in read-
only mode, and copies the data to the new database.
123. If the original database was in read-only mode, the new database
is set to read-only.
The original database is renamed by appending a string to the
end of the database name.
The new database is assigned the original database name:
Finance.
After the database administrator has confirmed that the new database is
operating correctly, the original database can be archived.
124. The DAC does not include the data inside your database. For
deployment best practices, you should have any necessary data
(configuration tables, basic lookup tables) already scripted out as
part of your deployment strategy. With the DAC approach, it makes
sense to put these scripts inside the database as objects.
For example, you might have a stored procedure called usp_deploy
that populates all of the necessary configuration tables via insert
statements.