Andreas Grabner maintains that most performance and scalability problems don’t need a large or long running performance test or the expertise of a performance engineering guru. Don’t let anybody tell you that performance is too hard to practice because it actually is not. You can take the initiative and find these often serious defects. Andreas analyzed and spotted the performance and scalability issues in more than 200 applications last year. He shares his performance testing approaches and explores the top problem patterns that you can learn to spot in your apps. By looking at key metrics found in log files and performance monitoring data, you will learn to identify most problems with a single functional test and a simple five-user load test. The problem patterns Andreas explains are applicable to any type of technology and platform. Try out your new skills in your current testing project and take the first step toward becoming a performance diagnostic hero.
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
Become a Performance Diagnostics Hero
1.
T21
Performance
Testing
10/6/16
15:00
Become
a
Performance
Diagnostics
Hero
Presented
by:
Andreas
Grabner
Dynatrace
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/
2.
Andreas
Grabner
Performance
enthusiast
Andreas
Grabner
has
been
a
developer,
tester,
architect,
and
product
evangelist
for
the
past
fifteen
years
for
several
testing
and
diagnostics
companies
including
Segue,
Borland,
Compuware,
and
Dynatrace.
Andreas
now
helps
organizations
find
performance,
scalability,
and
architectural
problems
in
their
applications.
Speaking
at
meetups,
user
groups,
and
international
conferences,
he
shares
his
knowledge
and
teaches
others
how
to
recognize
and
avoid
the
formerly-‐mentioned
problems.
3. Become a Performance
Diagnostics Hero
Without running large scale load tests
Andreas Grabner: @grabnerandi, andreas.grabner@dynatrace.com
http://www.slideshare.net/grabnerandi
5. DevOps @ Target
presented at Velocity, DOES and more …
http://apmblog.dynatrace.com/2016/07/07/measure-frequent-successful-software-releases/
“We increased from monthly to 80
deployments per week
… only 10 incidents per month …
… over 96% successful! ….”
31. cite the database as the most
common challenge or issue
with application performance
88%
32. Confidential, Dynatrace, LLC
Inefficient Loading of Too Much Data!
SQL Heavy: 2111 SQL
Calls, 6.97s Exec TimeADO.NET: Most of this time is
reading and processing SQL Results
Garbage Collection:
impacting response time
Resource Impact: CPU
& Memory due to
Execution Plan
Generation
33. Confidential, Dynatrace, LLC
N+1 Query Problem Pattern
N+1 Query: Same SQL
called 359! times
N+1 Query: Even with the same
Bind Values called 26! times
34. Confidential, Dynatrace, LLC
The Impact on SQL Servers Execution Plan Cache
CPU intensive operation:
A new execution plan is generated for
each unique SQL statement
The plan cache is used to save all the
execution plans in case they can be reused
36. Confidential, Dynatrace, LLC
SQL Server Perf Metrics: Requests vs Compilations
Ratio: More than 50% of the queries
(Batch Requests/s) requires the generation
of a new execution plan (Compilation/s)
High CPU caused by
execution plan generation
High Disk I/O Operations
because the plan cache is
using memory “stolen”
from the buffer cache
38. Resource Improvements with Parameterized Queries
Throughput improved by half! High
number of queries processed but
low number of compilations
Disk I/O reduced by more
than half as there is more
memory available to cache
the data in memory
Reduced CPU usage
39. Key App Metrics
# of SQL Calls per Request
# of same SQL Execs (1+N)
Rows/Data Transferred
Ratio Batch Requests to
Compilations
44. #4: Excessive Thread Usage per Request
Excessive Remoting:
40! RMI Calls
on 20! parallel threads
Excessive SQL: 2790!
SQL Calls Total
Worker Thread Exhaustion:
Excessive use of threads in App Server also
causes backup of requests in Web Server.
Watch your Worker Thread Pools!
45. Tip: Follow Threads through PurePath
Thread Name:
Count Unique Thread IDs
Async/Sync Nodes:
RMI calls done on
background threads
46. Tip: Watch out for Sync / Wait
1.63s in Object.wait
Means that this thread is put to hold
Waiting on the next
Connection to become
available!
47. Key Metrics
Pool and Queue Sizes
Time in Sync & Wait
# of Threads per Request
Balanced Requests
49. #1: Log Hotspots in Frameworks!
callAppenders clear CPU and I/O Hotspot
Excessive logging through Spring Framework
50. #2: Debug Log and outdated log4j library
#1: Top Problem: log4j.callAppenders
-> 71% Sync Time
#2: Most of logging done from
fillDetail method
#3: Doing “DEBUG” log
output: Is this necessary?
51. #3: Overhead caused by Exceptions
fillInStackTrace is Top 2 in CPU Hotspots
All these Exceptions that never show up in
a log file are consuming all CPU
52. Identify “hidden” Technical Debt
2-5 Log Messages per 5 Min
Looking at the important
(SEVERE, FATAL, …) log messages
written
Up to 20000 Custom Exceptions
That’s about 4000x the number
of Exceptions per Log Message
53. Key Metrics
# of Log Entries
Size of Logs per Use Case
Ratio Exceptions to Logs
55. Online Sports Club Search Service
2015201420xx
Response Time
2016+
1) Started as a
small project
2) Slowly growing
user base
3) Expanding to
new markets –
1st performance
degradation!
4) Adding more markets
– performance becomes
a business impact Users
4) Potentially start
loosing users
56. Can‘t scale vertically endlessly!
2.68s Load Time
94.09% CPU
Bound
Early 2015: Monolithic App
58. 7:00 a.m.
Low Load and Service running
on minimum redundancy
12:00 p.m.
Scaled up service during peak load
with failover of problematic node
7:00 p.m.
Scaled down again to lower load
and move to different geo location
Testing the Backend Service alone scales well …
62. 26.7s Load Time
5kB Payload
33! Service Calls
99kB - 3kB for each call!
171!Total SQL Count
Architecture Violation
Direct access to DB from frontend service
Single search query end-to-end
63. 2.5s (vs 26.7)
5kB Payload
1! (vs 33!) Service Call
3kB (vs 99) Payload!
3!(vs 177) Total
SQL Count
The fixed end-to-end use case
“Re-architect” vs. “Migrate” to Service-Orientation
64. Key Metrics
# of Service Calls
Payload of Service Calls
# of Involved Threads
1+N Service Call Pattern!