Stress test SQL Server with OStress - bitch!

1. Thank you to our Sponsors
Breaking Data – Terry Bunio
@Tbunio
Media Sponsor:

2. When Good Data Goes Bad…

3. Who Am I?
• Terry Bunio
• Data Base Administrator
- Oracle, SQL Server 6,6.5,7,2000,2005,2012, Informix, ADABAS
• Sharepoint fan
• Data Modeler/Architect
- Investors Group, LPL Financial, Manitoba Blue Cross, Assante
Financial, CI Funds, Mackenzie Financial
- Normalized and Dimensional
• Agilist
- Innovation Gamer, Team Member, SQL Developer, Test writer,
Sticky Sticker, Project Manager, PMO on SAP Implementation

9. My Blog – www.agilevoyageur.com

10. Breaking Data

11. SQL Saturday Winnipeg
• November 22nd – Red River Community College
- Downtown Campus
• First SQL Saturday ever in Winnipeg
• 3rd in Canada after Toronto and Vancouver
• 20 Sessions
• 4 Tracks
- Business Intelligence
- DBA
- Developer
- New Database Technology

12. March 2 – 3, 2015
Call for Speakers Open!
www.prairiedevcon.com

13. Question?
• What is broken data?
• How do we fix it?

14. Objectives
• Mine
- Hopefully introduce a couple of ideas you can take back and
improve on
• Yours?

16. Three types of broken data
• Inconsistent - Easy
• Incoherent - Moderate
• Ineffectual - Hard

17. Inconsistent

18. Inconsistent
• All Data must have a structure
• Domain
- “In data management and database analysis, a data
domain refers to all the values which a data element may
contain.” - Wikipedia

19. Inconsistent
• Domains are a simple way to ensure data consistency
• Many times this is overlooked due to tools that don’t
promote it
- Hand-rolled SQL DDL and scripts
• Use tools that require you to define Data Domains
- Erwin
- Oracle SQL Data Modeler
• FREE!
• http://www.oracle.com/technetwork/developer-tools/
datamodeler/overview/index.html

20. Inconsistent

21. Inconsistent
• You try and find inconsistencies in that model!
• Luckily I have used Oracle Data Modeler and defined the
following Domains

23. Incoherent

25. Incoherence

26. Incoherent
• Many databases remain coherent by using Foreign Key
Constraints
- These constraints ensure records can’t be stored in one table
unless the row they refer to in another table already exists
- These constraints are usually enabled all the time
- They can slow down performance and cause the data to be
Ineffectual

27. Incoherent
• Most databases create the constraints and leave them
enabled all the time
• Constraint Double-Whammy
- Slows down actual inserted/modification of data
- Further slows down code as you validate the code values before
you try to insert/update to avoid throwing database exception

28. Incoherent
• Alternative approach
- Leave Constraints disabled
- Attempt to re-enable them periodically to report on any invalid
data – Daily or Weekly
• You can then correct that data
- Disable constraints again
• In the past this process wasn’t possible due to the length
of time such a process would take
• only takes 75 minutes to re-enable 616 Foreign Key
constraints on a 1.1 Terabyte MSSQL 2012 database.
Thanks Microsoft!

29. Incoherent
• Demo

30. Ineffectual
• There are three types of database Performance Tuning
that you can do to make your data less ineffectual
- Execution Plan / Statistics IO
- SQL Profiler
- OStress

31. Execution Plan
• Demo

32. Execution Plan
• 1.sql

34. Execution Plan
• You then get an Execution Plan tab
• Execution Plan process has actually got very good at
recommending indexes.
• Anyone remember MSSQL Index Wizard?

38. How to read Execution Plan
• Index Seek >> Index Scan >> Table Scan
• Look for steps that are a large percentage of the overall
query
- See if those steps are using the right indexes
• Hover over each step to get details

39. How to read Execution Plan
• Cached plan size – how much memory the plan
generated by this query will take up in stored procedure
cache. This is a useful number when investigating cache
performance issues because you'll be able to see which
plans are taking up more memory.
• Estimated Operator Cost – Overall percentage cost of
the step

40. How to read Execution Plan
• Estimated Subtree Cost – tells us the accumulated
optimizer cost assigned to this step and all previous
steps, but remember to read from right to left. This
number is meaningless in the real world, but is a
mathematical evaluation used by the query optimizer to
determine the cost of the operator in question; it
represents the amount of time that the optimizer thinks
this operator will take.
• Estimated number of rows – calculated based on the
statistics available to the optimizer for the table or index
in question.

41. SET STATISTICS IO ON

42. SET STATISTICS IO ON
• DBCC FREEPROCCACHE

44. SQL Profiler
• Demo

45. SQL Profiler

47. SQL Profiler

48. SQL Profiler

49. SQL Profiler

51. SQL Profiler
• You can save traces and replay those traces to simulate
load
• There are some limitations though
- Replay RPC events as remote procedure calls
- Replay attention
- Replay DTC transactions
- Replay as part of an automated scripts === SCALABLE Tool

53. OStress
• Comprised of two utilties:
- Read80Trace
• Required in order to convert trace files into RML files
- OSTRESS
• Multithreaded ODBC-based query utility. The OSTRESS utility reads input
from a command-line parameter. The command-line parameter can be an
RML file that is produced by the Read80Trace utility or a standard go-delimited
.SQL script file. In stress mode, one thread is created for each
connection, and all threads run as fast as possible without synchronization
among the threads. You can use this mode to generate a specific type of
stress load on the server.

54. Ostress
• First we need to download and install the tools on the
server where we want to run our trace files
- http://www.microsoft.com/en-us/
search/Results.aspx?q=ostress&form=DLC

55. Ostress
• Demo

58. Convert Trace Files to RML files
• DOS Command
- CD c:Program FilesMicrosoft CorporationRMLUtils
- ReadTrace –Ic:TraceFilesTraceSample.trc –oc:RMLFiles –T28
• T28 flag
- Important as it allows you to replay the RML file against SQL
Server

59. Ostress
• Now you can simply run Ostress with those RML files
- OSTRESS -creplay.ini -mreplay -T88 –ic:RMLFiles*rml –
oc:RMLFilesReplayResult

60. Specific use
• You can run and compare Ostress results when you
upgrade SQL Server or other system software and
hardware!
• You can compare them using the following command:
- ReadTrace –Ic:TraceFiles*.trc –oc:TraceFilesReplayResult –
dods –f
• You can answer confidently whether the new server can
handle the current production load and stress

62. Review
• The power of this structure is that we can now automate
hundreds of threads to replay loads on the database
• This can now also become part of automated
testing/continuous integration processes

63. Whew…

64. Questions?