wuqiSpank.org -- The First "Table Centric" Approach to Solving SQL Performance Problems

wuqiSpank.org / Slide 1 / STLCMG
wuqiSpank.org

wuqiSpank.org
erik.ostermueller@fisglobal.com
eostermueller@gmail.com
Performance Center of Excellence
at FIS -- #1 in the FinTech100
http://www.idc.com/downloads/Final_Rankings.pdf

Why
wuqiSpank?

Who needs wuqiSpank?
We should be able to code without this kind of an effort:

We should be able to code without these kinds of
obvious performance defects – 2,256 SQL invocations
with a single button click.

The #1 Server Side Perf Problem
Sampling of Performance Defects
● 2,272 SELECTs to table [BLAH] (single button click)
● The code is querying [TABLE BLAH] 15 times a second for data the rarely, if ever, changes
● [TABLE BLAH] hit 37k times for 250 logons.
● Both of these SQL are executed 548,976 times in a 15 minutes duration
● 82 sql stmts with single button click. Many of the tables are accessed repeatedly during the request.Here are
counts of times each table is accessed with a SELECT:
11 – BLAH
11 – OTHER_BLAH
16 -- BLAH_BLAH

Other examples (industry 1)
● 2003 -- Martin Fowler
● “Try to pull back multiple rows at once.
In particular, never do repeated queries
on the same table to get multiple rows.”
http://www.informit.com/articles/article.aspx?p=30661&seqNum=3

Martin Fowler says:
Never do repeated queries on the same table.
Does this graph help you any? No.

Martin Fowler says:
Never do repeated queries on the same table.
Does this SQL text help you any? No.
select custom0_.TC_9RUI as J1_84_, custom0_.TC_9RVT as J2_84_, custom0_.TC_9RWT as J3_84_, custom0_.XC_RAPI as J4_84_, custom0_.TC_9SZI as J5_84_,
custom0_.TC_9SCI as J6_84_, custom0_.XC_CVAC as J7_84_, custom0_.TC_9SGL as J8_84_, '10' as Pormula0_, custom0_.TC_9SCI as Pormula1_ FROM
XRTADLS.XC_CUMS custom0_ where custom0_.TC_9RUI in ('00000117') order by custom0_.TC_9RVT||custom0_.TC_9RWT;select Rankor0_.XC_NOPC as J1_121_0_,
Rankor0_.XC_NOGI as J2_121_0_, Rankor0_.XC_NOHT as J3_121_0_, Rankor0_.XC_CYWT as J4_121_0_ FROM XRTADLS.XC_CUSTM Rankor0_ where
Rankor0_.XC_NOPC='10' and Rankor0_.XC_NOGI='AT_OPPICER';select adeRus0_.WRKDWNC as WRKDWNC138_, adeRus0_.WRKDWOC as WRKDWOC138_,
adeRus0_.WRKDWPI as WRKDWPI138_, adeRus0_.WRKDWQT as WRKDWQT138_, adeRus0_.WRKDWRC as WRKDWRC138_, adeRus0_.WRKDWST as WRKDWST138_
FROM XRTADLS.WRKGNER adeRus0_ where adeRus0_.WRKDWNC='SECURITY_USERS' and adeRus0_.WRKDWOC=' ' and adeRus0_.WRKDWPI='PARIS';select
mst0_.XC_RAAI as J1_88_0_, mst0_.XC_RACI as J2_88_0_, mst1_.XC_RPAI as J1_90_1_, mst1_.XC_RPCI as J2_90_1_, mst1_.XC_RPDI as J3_90_1_, mst1_.XC_CQJC as
J4_90_1_, mst1_.XC_RPEI as J5_90_1_, mst1_.XC_RPPC as J6_90_1_, mst1_.XC_RPGI as J7_90_1_, mst0_.XC_CQYI as J3_88_0_, mst0_.XC_CQZI as J4_88_0_,
mst0_.XC_CRAI as J5_88_0_, mst0_.XC_RARI as J6_88_0_, mst0_.XC_RAGT as J7_88_0_, mst0_.XC_RAH8 as J8_88_0_, mst0_.XC_RAN8 as J9_88_0_, mst0_.XC_RAO8 as
J10_88_0_, mst0_.XC_RAZC as J11_88_0_, mst0_.XC_GIZI as J12_88_0_, mst0_.XC_CROI as J13_88_0_, mst0_.XC_CRCC as J14_88_0_, mst0_.XC_CRDC as J15_88_0_,
mst0_.XC_RDRC as J16_88_0_, mst0_.XC_RADI as J17_88_0_, mst0_.XC_CRRC as J18_88_0_, mst0_.XC_CRPI as J19_88_0_, mst0_.XC_RRGI as J20_88_0_,
mst0_.XC_RRHI as J21_88_0_, mst0_.XC_RRDI as J22_88_0_, mst0_.XC_RRNI as J23_88_0_, mst0_.XC_RRPI as J24_88_0_, mst1_.XC_RPS8 as J8_90_1_, mst1_.XC_RPXL
as J9_90_1_, mst1_.XC_RGQL as J10_90_1_, mst1_.XC_RPRI as J11_90_1_, mst1_.XC_RPMC as J12_90_1_, mst1_.XC_RGA_ as J13_90_1_, mst1_.XC_RGR_ as J14_90_1_,
mst1_.XC_RGC_ as J15_90_1_, mst1_.XC_RGD_ as J16_90_1_, mst1_.XC_RGOP as J17_90_1_, mst1_.WRKDUR8 as WRKDUR18_90_1_, mst1_.WRKDUQ8 as
WRKDUQ19_90_1_, mst1_.XC_RPVC as J20_90_1_, mst1_.XC_CQKI as J21_90_1_, mst1_.XC_CQLT as J22_90_1_, mst1_.XC_CQMN as J23_90_1_, mst1_.XC_RAKT as
J24_90_1_ FROM XRTADLS.XC_CAMS mst0_, XRTADLS.XC_ORDER mst1_ where mst0_.XC_CRCC=mst1_.XC_CQJC and mst0_.XC_RAAI=mst1_.XC_RPAI and
mst0_.XC_RACI=mst1_.XC_RPCI and mst1_.XC_RPEI='00' and mst1_.XC_RPPC='00' and (mst0_.XC_RARI='00000117' or mst0_.XC_CQZI='00000117') and
mst1_.XC_RPVC='A';
select acctma0_.TC_9RSI as J1_100_0_, acctma0_.TC_9RTI as J2_100_0_, acctmi1_.TC_9RSI as J1_101_1_, acctmi1_.TC_9RTI as J2_101_1_, acctac2_.TC_9EYI as
J1_102_2_, acctac2_.TC_9EZI as J2_102_2_, acctac2_.TC_9PAI as J3_102_2_, acctac2_.TC_9PRI as J4_102_2_, acctac2_.XC_CJMC as J5_102_2_, acctma0_.TC_9RWI as
J3_100_0_, acctma0_.TC_9RXI as J4_100_0_, acctma0_.XC_CMKI as J5_100_0_, acctma0_.XC_CMMI as J6_100_0_, acctma0_.XC_CMLI as J7_100_0_, acctma0_.TC_9RUI as
J8_100_0_, acctma0_.TC_9CH8 as J9_100_0_, acctma0_.TC_9CI8 as J10_100_0_, acctma0_.TC_9CJ8 as J11_100_0_, acctma0_.TC_9CK8 as J12_100_0_,
acctma0_.XC_DHD8 as J13_100_0_, acctma0_.TC_9CPC as J14_100_0_, acctma0_.XC_AWAC as J15_100_0_, acctma0_.TC_9EP8 as J16_100_0_, acctma0_.XC_GJAI as
J17_100_0_, acctma0_.XC_CMPI as J18_100_0_, acctma0_.TC_9RVL as J19_100_0_, acctma0_.TC_9EE_ as J20_100_0_, acctma0_.TC_9EP_ as J21_100_0_,
acctma0_.TC_9EG_ as J22_100_0_, acctma0_.XC_CMS_ as J23_100_0_, acctma0_.XC_CNPC as J24_100_0_, acctma0_.XC_CNGC as J25_100_0_, acctma0_.XC_DUPC as
J26_100_0_, acctma0_.XC_CMOC as J27_100_0_, acctma0_.TC_9CQI as J28_100_0_, acctma0_.TC_9CPI as J29_100_0_, acctma0_.TC_9CRI as J30_100_0_,
acctma0_.TC_9COI as J31_100_0_, acctma0_.TC_9DLC as J32_100_0_, acctmi1_.XC_DUEC as J3_101_1_, acctmi1_.TC_9CCC as J4_101_1_, acctmi1_.TC_9CRC as
J5_101_1_, acctmi1_.XC_EGPI as J6_101_1_, acctmi1_.XC_EJY8 as J7_101_1_, acctac2_.TC_9PM8 as J6_102_2_, acctac2_.TC_9PEI as J7_102_2_, acctac2_.XC_RLJC as
J8_102_2_, acctac2_.TC_9HK_ as J9_102_2_, acctac2_.TC_9HL_ as J10_102_2_, acctac2_.TC_9HM_ as J11_102_2_, acctac2_.TC_9HN_ as J12_102_2_, acctac2_.WRKCVVA
as WRKCVVA102_2_, acctac2_.WRKCZCA as WRKCZCA102_2_, acctac2_.WRKDEIA as WRKDEIA102_2_, acctac2_.WRKECKA as WRKECKA102_2_, acctac2_.XC_CKU_ as

2010 -- “Too many Database Calls”
#1 in DynaTrace's top 10 perf problems
http://apmblog.compuware.com/2010/06/15/top-10-performance-problems-taken-from-zappos-monster-and-co/

2012 -- “Too many external system calls are
performed in a synchronous and sequential manner.”
#4 in DZone's top 10 perf problems
http://apmblog.compuware.com/2010/06/15/top-10-performance-problems-taken-from-zappos-monster-and-co/

2013 -- #1 server-side problem
(DynaTrace again)
http://apmblog.compuware.com/2013/04/10/top-8-application-performance-landmines/

2013 “SELECT N+1”
Part of AppDynamic's "Common Application
Problem" series
http://www.appdynamics.com/blog/apm/common-application-problems-and-how-to-fix-them-the-select-n-1-problem/

Jack Shirazi, author of “Java Performance Tuning”
says “There are ways in which you can paint
yourself in a corner so that the only alternative is to
throw away the application and start again”
http://www.javaspecialists.eu/archive/Issue066.html

The Cost of Retreat is High

Demo

What is
wuqiSpank?

What is so special?
● The first Table-Centric Visualization
● Visually identify repetitious patterns
● Navigate the Perilous Voyage from
* Small development database
* To gargantuan producticon database

How do find root cause?
● Familiarize yourself with known
SQL performance anti-patterns
● Then provide a tool (wuqiSpank) that makes it easy
to see anti-patterns
● Apply a known solution

Anti-Pattern #1: SELECT N+1
● Code iterates through the result set of the parent
table and executes SELECTs to the child table. The
extra queries incur unnecessary overhead.
ResultSet parentRs =
myStatement.executeQuery(strParentSELECT);
while(rs.next()) {
String myId = rs.getString(1);
ResultSet childRs = statement.executeQuery(
"select * from child where id = " + myId);
//do something with childRs
}

● Known Solution: Reduce query count by joining parent and child
table.

http://blog.jooq.org/tag/object-relational-mapping/
http://stackoverflow.com/tags/select-n-plus-1/hot
http://use-the-index-luke.com/sql/join/nested-loops-join-n1-problem
http://thecuttingledge.com/?p=169
http://ayende.com/blog/1328/combating-the-select-n-1-problem-in-nhibernate
http://pramatr.wordpress.com/2009/02/05/sql-n-1-selects-explained/
http://www.hibernatingrhinos.com/products/nhprof/learn/alert/selectnplusone
http://www.pbell.com/index.cfm/2006/9/17/Understanding-the-n1-query-problem
http://dotnet.dzone.com/news/select-n1-problem-%E2%80%93-how
http://java.dzone.com/articles/martin-fowler-orm-hate
http://learningviacode.blogspot.com/2012/07/the-n-1-select-problem-in-hibernate.html
http://learningviacode.blogspot.com/2012/08/fetch-join-for-collection.html
http://learningviacode.blogspot.com/2012/08/fetch-subselect.html
http://learningviacode.blogspot.sg/2012/08/batch-size-n.html
http://learningviacode.blogspot.in/2012/07/lazyextra.html
http://stackoverflow.com/questions/97197/what-is-the-n1-selects-issue
http://docs.telerik.com/data-access/developers-guide/profiling-and-tuning/profiler-and-tuning-advisor/data-access-profiler-n-plus-one-
problem

Anti-Pattern #2: Uncached Static Data
● Definition of static table: less than 1
INSERT/UPDATE/DELETE an hour
● Source code repeatedly SELECTs static data, even
though that data rarely changes. The extra queries
incur unnecessary overhead.

Anti-Pattern #2: Uncached Static Data
Known Solution:
● Reduce query count with use EHCache or similar
caching facility to retrieve data from memory
instead of from disk

Anti-Pattern #3: Uncached Dynamic
Data
Source code retrieves dynamic data from db and re-
queries that same data before the business process
has completed. The extra queries incur unnecessary
overhead.

Anti-Pattern #3: Uncached Dynamic
Data
Known Solution: Reduce query count by
temporarily storing the initial query results so they
are available to subsequent code.
● For example, say that two web service calls are
made after a user clicks a button. Both calls require
the same five SQL queries for initialization.
Solution: Refactor code into a single web service
call instead of two.

Table Roles
● Parent and Child

Table Roles
● Teams 'flag' certain tables as
'growth' tables – see red circle?
● large row counts like TRAN_HIST
● Reminder for developers to avoid
multiple queries to these tables.
Voyage from small to large.

Quiz
What happens when we move
functionally-complete code
from a small development database
to a large production database?

Quiz: Answer
The software developers go on vacation.

Table Roles
● Teams 'flag' certain tables as
'static' tables – see blue circle?
● For data that changes seldom.
Perhaps once an hour or less.
● Reminders for developers to
provide caching

Table Roles
Patterns

Root Cause - Summary
● We find 'Root Cause' for a defect by identifying the
roles the tables play in known anti-patterns
● Parent
● Child
● Growth - Red
● Static – Blue, like frozen, static glacier

Why Bother?

Chunky Outperforms Chatty

Chunky Outperforms Chatty
The performance problems with chatty code are easy
to demonstrate:
http://ostermueller.blogspot.com/2014/10/
chunky-outperforms-chatty.html

SQL Sequence Diagrams
wuqiSpank does Data access strategy comparisons
● Scenario 5 is the chattiest
● Scenario 1 is the chunkiest
● 2, 3 and 4 line up inbetween

Premature Optimization
Some early tuning is warranted when:
● When you’re addressing the worst server-side
performance problem in enterprise software
● When the cost of retreat is high.

Premature Optimization
Some early tuning is warranted when:
● When the problem is easy to demonstrate
● When the response of development teams to identify
the problem has been meager
● Effort required to catch worst problems: trivial

SPE Whining and Complaining: 1
● We've got the wrong staffing model for
performance.
● Why are tiny SPE teams cleaning up huge messes
left by much larger development teams?

● Don't quite have the right tools.
No tool visibility into the 1 month of work done by
a developer on a SQL strategy.

● Performance is placed in the wrong part of the
SDLC
● That last lead-up segment of the plan before
production
● --> that was meant for LOW RISK activities
● --> It was not meant for "rewrite it all because we
got a late start on performance."

Conclusion
wuqiSpank is an open-source SQL
tracing/visualization tool for software teams.

Conclusion
It provides a Missing Road Map for Refactoring
SQL
...saving many hours of research when a
performance defect is opened.

Conclusion
Chunky outperforms Chatty
We already know the chatty pattern doesn't perform.
Why wait to optimize when the cost to fix is so low?

Conclusion
Some platforms are more equal than others.
wuqiSpank reads SQL from any platform, but has
very nice live tracing for Java SQL (JDBC).

Conclusion
A safer Voyage from Small-to-Large database

Conclusion
The exact line of code that executed the SQL
....but don't get wrapped up in that.

Conclusion
Retreat has varying degrees of difficulty

Conclusion
Yes we must avoid Premature Optimization.
● But it does not excuse developers from all
performance concerns.
● We Must Finally Draw the line
and start making a dent on the biggest server-side
performance problem.

Thank You
erik.ostermueller@fisglobal.com
eostermueller@gmail.com

Help

Help
Submit wuqiSpank blog entry to one of these sites:
LinkedIn: Software Performance Engineering
LinkedIn: Software Test & Performance Group
LinkedIn: Performance Specialists

Help
Provide server to host an internet discussion group
● Review wuqiSpank on your blog.
● Submit wuqiSpank blog entry to one of these sites:
practicalperformanceanalyst.com
javaperformancetuning.com
softwaretestpro.com
associationforsoftwaretesting.org

Help
Vote on new Features:
● Security
● Move from Create Commons Non-Commercial
● Angular JS UI with Infinite Scroller
● "Tuning Recommendations"
● Generate JMeter load project with SQL

Help
Coding
● Java - Command line program to connect intrace-
agent.jar
● Multi-threaded issues
● 1) https://github.com/eostermueller/headlessInTraceClient/issues/2
● AngularJS Guru to build new prototype for UI

Therefore, consider a situation where you need to
get 50 people that you can identify by a primary key
in your domain model, but you can only construct a
query such that you get 200 people, from which
you'll do some further logic to isolate the 50 you
need. It's usually better to use one query that brings
back unnecessary rows than to issue 50 individual
queries.
--Martin Fowler
http://www.informit.com/articles/article.aspx?p=30661&seqNum=3

wuqiSpank.org -- The First "Table Centric" Approach to Solving SQL Performance Problems

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a wuqiSpank.org -- The First "Table Centric" Approach to Solving SQL Performance Problems

Semelhante a wuqiSpank.org -- The First "Table Centric" Approach to Solving SQL Performance Problems (20)

Último

Último (20)

wuqiSpank.org -- The First "Table Centric" Approach to Solving SQL Performance Problems