This presentation was given as part of a Dynatrace Lunch & Learn event. APM (=Application Performance Management) allows us to transform the way we develop, deploy and run software. Here are some ideas how APM can be (r)evolutionized

1. Dynatrace Lunch & Learn
1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

2. Dynatrace
• 5,700 customers, from Born-In-The-Cloud to Global 2000
• Gartner Market Share leader for two consecutive years
• Used by 17 of the top 20 online retailers - Internet RETAILER, 2014 TOP 500 Guide
• Net Promoter Score of 89, best NPS among all APM vendors
• 600 R&D and 300 technical experts - Largest expertise in APM
2 @Dynatrace

3. (R)Evolutionize APM
- by learning from DevOps Movement
- More on http://blog.dynatrace.com
Hosted by: Andreas Grabner - @grabnerandi
3 @Dynatrace

4. 4 @Dynatrace

5. 5 @Dynatrace

6. 6 @Dynatrace

7. The stuff we did
when we were a Start Up
and we All were
Devs, Testers and Ops
7 @Dynatrace

8. The Idea is to …
Improve Collaboration between Developers and Operators and
• Stop wasting money
• Stop wasting time
• Stop frustrating people
• Start building great systems
•And do it much faster and more frequent!
8 @Dynatrace

9. Who made the transition?
700 Deployments / Year
10+ Deployments / Day
50-60 Deployments / Day
Every 11.6 seconds
9 @Dynatrace

10. Inside the Amazon Numbers!
Deploying every 11.6s
75% fewer outages since 2006
90% fewer outage minutes
~0.001% of deployments cause a problem
Instantaneous automatic rollback
10 @Dynatrace

11. Recommended Book
It’s a culture thing
11 @Dynatrace

12. Status Quo: Unreliable Software Impacts Business
12 @Dynatrace

13. The “War Room”
Facebook – December 2012
13 @Dynatrace

14. YES we know this
80%Dev Time in Bug Fixing
$60B Defect Costs
BUT
~80% of problems
caused by ~20% patterns
14 @Dynatrace

15. 4 Situations on
WHY this happened,
HOW to avoid it
Metrics to look at
15 @Dynatrace

16. 16 @Dynatrace

17. #Push
without a Plan
17 @Dynatrace

18. Mobile Landing Page of Super Bowl Ad
434 Resources in total on that page:
230 JPEGs, 75 PNGs, 50 GIFs, …
Total size of ~
20MB
18 @Dynatrace

19. Metrics: Load Time,
# Resources (Images, …),
# HTTP 3xx, 4xx, 5xx
Dev: Build for Mobile
Test: Test on Mobile
Ops: Monitor Mobile
19 @Dynatrace

20. 20 @Dynatrace

21. #“Blindly”
(Re)use Existing
Components
21 @Dynatrace

22. Requirement: We need a report
22 @Dynatrace

23. Using Hibernate results in 4k+ SQL Statements to
display 3 items!
Hibernate
Executes 4k+
Statements
Individual
Execution VERY
FAST
But Total SUM
takes 6s
23 @Dynatrace

24. Metrics: # Total SQLs
# SQLs / Web Request
# Same SQLs / Request
Transferred Rows
Dev: “Learn” Frameworks
Test: With realistic Data
Ops: Monitor Usage
24 @Dynatrace

25. 25 @Dynatrace

26. Don’t assume
You know the
environment
26 @Dynatrace

27. Distance Calculation Issues
480km biking
in 1 hour!
Solution: Unit Test in
Live App reports Geo
Calc Problems
Finding: Only
happens on certain
Android versions
27 @Dynatrace

28. 3rd Party Issues
Impact of bad
3rd party calls
28 @Dynatrace

29. Metrics: Crashes, Exceptions, # and
Status of 3rd Party Calls, Payload of
Web Service Calls
Dev: Build for Mobile
Test: Test on Mobile and Diff. Carriers
Ops: Monitor Mobile
29 @Dynatrace

30. 30 @Dynatrace

31. #No “Agile”
Deployment
31 @Dynatrace

32. Load Spike resulted in Unavailability Ad on air
32 @Dynatrace

33. Alternative: “GoDaddy goes DevOps”
1h before
SuperBowl KickOff
1h after
Game ended
33 @Dynatrace

34. Behind the Scenes
34 @Dynatrace

35. Metrics: Availability
Page Size, # Objects
# Hosts, # Connections
DevOps: “Feature” Switches
35 @Dynatrace

36. Unit & Integration Tests
Commit Stage
• Compile
• Execute Unit Test
• Code Analysis
• Build installers
Automated
Acceptance
Testing
Performance Tests
Automated
Capacity
Testing
Production
Monitoring
Manual testing
• Key showcases
• Exploratory testing Release
Functional Tests
36 @Dynatrace

37. The Dynatrace APM Trifecta
Deep Dive
• Deepest visibility
• All transactions
• Discrete transaction tracing
• Code-level details
• Full lifecycle integration
Synthetics
• SaaS Delivery
• World Presence
• Running in minutes
• Worldwide network
• Web & mobile
• Enterprise Apps
Passive
Network
• Real User Monitoring
• Passive
• All Apps
• Tier-by-tier
• Network visibility
Dynatrace
APM
• Development • Test • Production • Business
SaaS Delivery Model
• Internet Synthetic
• Industry Benchmarks
• dynaTrace DTM
• User Experience
• Mobile
On Prem Delivery Model
• DCRUM
• dynaTrace DTM
• Synthetic Enterprise
• Real User Experience
• APM in the Lifecycle
37 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

38. Dynatrace Synthetic
Global Internet Transaction Management Deep Dive
•Deepest visibility
•All transactions
•Discrete transaction tracing
•Code-level details
•Full lifecycle integration
Synthetics
•SaaS-delivered
•Running in minutes
•Worldwide network
•Web & mobile
Passive
•Real Users
•Passive
•All Apps
•Tier-by-tier
•Network visibility
38 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

39. The World’s Most Comprehensive Performance and
Testing Network
Synthetic Agents
Deployed and managed
by Dynatrace
Backbone Last Mile Cloud
High Volume Load
Generation
13 Global locations
Web Performance
Management
100+ locations
Web Performance
Management and
Load Testing
10,000’s+ locations
Mobile
Dozens of countries
connected to real
wireless carriers
Synthetic Private Agents
Deployed by Customer
• SaaS Private Agents
• SaaS Private Peers
• Enterprise Synthetic (OnPrem)
39 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

40. APM Portal - Out of The Box Dashboards
Critical
Transactions
3rd Party
Errors
Global Perspective
40 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

41. Automated Diagnostics and Drilldown to PurePath
Home page is slow
Event
Notification
Problem Identification
Automatic Problem
Analysis
1st Byte Time
41 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

42. Datacenter Monitoring
Passive Application Monitoring
Featuring
• Transaction Trace Analysis
Deep Dive
•Deepest visibility
•All transactions
•Discrete transaction tracing
•Code-level details
•Full lifecycle integration
Synthetics
•SaaS-delivered
•Running in minutes
•Worldwide network
•Web & mobile
Passive
•Real Users
•Passive
•All Apps
•Tier-by-tier
•Network visibility
42 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

43. How Does It Work?
• Fault Domain Isolation: Locate Bottlenecks
• Business Impact: Managing vs. Monitoring
• Prioritization: Affected Users, Affected Transactions
• All Applications: Across entire datacenter
• All Users: By User ID, By Locations, Citrix, VM, WAN, and
LAN
• Always On: Continuous data feed of Performance
43 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

44. Affected Users
Cross-Tier FDI
Centralized View of All Applications
Synthetic Network Health
Overlay Charts
44 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

45. Detailed Operation Breakdown
Client Delays
Shown Here
45 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

46. Dynatrace Application
Deep Transaction Monitoring Deep Dive
•Deepest visibility
•All transactions
•Discrete transaction tracing
•Code-level details
•Full lifecycle integration
Synthetics
•SaaS-delivered
•Running in minutes
•Worldwide network
•Web & mobile
Passive
•Real Users
•Passive
•All Apps
•Tier-by-tier
•Network visibility
46 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

47. Test Environment
Production Environment
8x slower
3x more SQL
47 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

48. Test Environment Production Environment
Hibernate,
Hibernate,
Classloading,
XML – The Key
Classloading, XML
– The Key Hotspots
I/O for Web
Requests Hotspots
doesn’t
even show up!
That’s Normal:
Having I/O for Web
Request as main
contributor
48 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

49. Top Contributor
Class.getInterfaces
Called from Hibernates
FieldInterceptionHelper
These calls all originate
form thousands of calls to
find item by code
49 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

50. Every 11.6 seconds
Unit & Integration Tests
Commit Stage
• Compile
• Execute Unit Test
• Code Analysis
• Build installers
Automated
Acceptance
Testing
Performance Tests
Automated
Capacity
Testing
Production
Monitoring
Manual testing
• Key showcases
• Exploratory testing Release
Functional Tests
50 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace

51. Andreas Grabner
Dynatrace Developer Advocate
@grabnerandi
http://blog.dynatrace.com
51 @Dynatrace