This presentation provides a brief overview of APM solutions for the Azure cloud computing platform. We discuss three challenges unique to cloud computing which APM can address, and we summarize which APM techniques can be applied in IaaS, PaaS, and SaaS application architectures. To illustrate APM techniques for IaaS and PaaS we look at a variety APM offers in the Azure marketplace, including Riverbed AppInternals, Microsoft Application Insights, and New Relic. To illustrate APM techniques for SaaS, we look at how SharePoint Online can be instrumented using JavaScript injection. This presentation was prepared and delivered by Ian Downard to the Portland Azure User Group on March 28th, 2016, in Portland Oregon.

1. Why and How to Monitor
Application Performance
in Azure
March 28th, 2016

2. © 2015 Riverbed Technology. All rights reserved. 2
Abstract:
This presentation provides a brief overview of APM solutions for the Azure cloud computing
platform. We discuss three challenges unique to cloud computing which APM can address, and
we summarize which APM techniques can be applied in IaaS, PaaS, and SaaS application
architectures. To illustrate APM techniques for IaaS and PaaS we look at a variety APM offers in
the Azure marketplace, including Riverbed AppInternals, Microsoft Application Insights, and
New Relic. To illustrate APM techniques for SaaS, we look at how SharePoint Online can be
instrumented using JavaScript injection. This presentation was prepared and delivered by Ian
Downard to the Portland Azure User Group on March 28th, 2016, in Portland Oregon.
Keywords:
Azure, APM, application performance, AppInternals

3. © 2015 Riverbed Technology. All rights reserved. 3
Optimizing the performance of distributed applications for 15 years.
Technical evangelist for Riverbed AppInternals.
And currently trying to learn everything
there is to know about Azure.
Twitter: @iandownard

4. © 2015 Riverbed Technology. All rights reserved. 4
Why APM in the cloud?
What APM features are
built into Azure?
Survey 3rd party APM:
Riverbed AppInternals,
New Relic, & AppInsights
APM constraints in IaaS /
PaaS / SaaS
APM demos for Azure
VMs, Azure Cloud
Services, Azure Web
Sites, SharePoint Online
What does APM cost?

5. © 2015 Riverbed Technology. All rights reserved. 5
Applications
Data
Firewall
Network
OS
Virtualization
Hardware
Applications
Data
Firewall Rules
Network
OS
Virtualization
Hardware
Applications
Data
Firewall Rules
Network
OS
Virtualization
Hardware
Applications
Data
Firewall Rules
Network
OS
Virtualization
Hardware
On Premise
Azure IaaS
(Virtual Machines)
Azure PaaS
(Cloud Services)
Azure SaaS
(“Web Sites”, O365)
Youmanage
Youmanage
YoumanageManagedbyMicrosoft
ManagedbyMicrosoft
ManagedbyMicrosoft
Youmanage

6. © 2015 Riverbed Technology. All rights reserved. 6
http://riverbed.com/blogs/AppInternals-APM-for-Azure.html
Why is APM important for Cloud Computing?

7. © 2015 Riverbed Technology. All rights reserved. 7
“Right Size”
Your Scale
3 reasons.
Mitigate
Unpredictable
Performance
Verify Cloud
Vendor SLAs
Matching capacity to
demand can save $
Sharing hardware
is risky
Trust but verify

8. © 2015 Riverbed Technology. All rights reserved. 8
What unpredictable
performance?
Mitigate
Unpredictable
Performance

9. © 2015 Riverbed Technology. All rights reserved. 9
Response times for queries sent
to EC2, AppEngine and Azure
varied by a factor of 20
depending on the service and
time of day the cloud was
accessed.
Stress Tests Rain on Amazon’s Cloud. IT News (August 2009)
http://www.itnews.com.au/news/stress-tests-rain-
on-amazons-cloud-153451

10. © 2015 Riverbed Technology. All rights reserved. 10
• Feb 18 and March 9 – Google
Compute Engine down for 1 hour
• March 11 – Apple iCloud down for 12
hours due to DNS error
• March 16 – 2 hour outage in Azure
IaaS and PaaS due to “network issue”
The 10 Biggest Cloud Outages Of 2015. CRN Magazine (August 2015)
http://www.crn.com/slide-shows/cloud/300077635/the-
10-biggest-cloud-outages-of-2015-so-far.htm

11. © 2015 Riverbed Technology. All rights reserved. 11
Our experience is that multiple
virtual machines can share CPUs
and main memory surprisingly
well in cloud computing, but that
network and disk I/O sharing is
more problematic.
A View of Cloud Computing. Communications of the ACM (April 2010).
Michael Armbrust, et al.

12. © 2015 Riverbed Technology. All rights reserved. 12
What’s the best
APM solution
out there?

13. © 2015 Riverbed Technology. All rights reserved. 13
One-size-fits-all
APM does not exist.
How do you weigh your options?

14. © 2015 Riverbed Technology. All rights reserved. 14
What do you need from APM?
Application maps
End-to-End
transaction tracing
Business
transaction metrics
Alerting UI
Easy to use UI
Web UI
Dashboard
Web page object
waterfall data
Central
configuration
management
Scalable
management server
On-Premise / SaaS
Cloud compatible
Ready to use out of
the box
Transaction search
Sampling vs.
Continuous tracing
Sampling frequency
Big-Data driven
analytics
Full transaction
history
Application
framework coverage
Queue and async
architecture support
End user
experience
monitoring
Try and buy
ALM Integration
Supports DevOps
Practices
Cross-tier
transaction
stitching
Low instrumentation
overhead
Export CSV raw
data
Export PDF reports
Feed Power BI,
Splunk, etc
Stream raw data to
Azure blob storage
Powerful analytics
engine
Insightful statistical
graphics
Support for
statistical computing
Instrumentation
SDK / API
Extensibility (roll
your own agent)
JavaScript injection
Agent based
instrumentation
Log Analytics
Cost
Open Source
Full stack monitoring
Kernel level tracing
Integration with
packet captures
Instruments native
code
Memory leak
detection
Broad adoption
Active user
community
CPU/Disk/Network
resource monitoring

15. © 2015 Riverbed Technology. All rights reserved. 15
Lets survey some APM
options in Azure.
Azure Native Azure Marketplace
• Azure Diagnostics
• AppInsights
• Logging
• Riverbed AppInternals
• Dynatrace Ruxit
• New Relic

16. © 2015 Riverbed Technology. All rights reserved. 16
Azure Diagnostics
 Enable during provisioning
 View data in the Azure Portal UI
 Save data in blob storage

17. © 2015 Riverbed Technology. All rights reserved. 17
Azure Diagnostics
 Configure metrics under VM details
 Raw data saved in table storage

18. © 2015 Riverbed Technology. All rights reserved. 18
 Azure Linux Agent (waagent) https://github.com/Azure/WALinuxAgent
– Pulls extensions and configurations at provision-time.
– Extensions, such as this one:
Microsoft.OSTCExtensions.LinuxDiagnostic
– Configures swap space, snapshooting, encryption, diagnostics, custom
post-provision scripts
– Pings a “wire” service for a new gold state every 25 seconds. Versions
determined at run-time.
– Runs under, /var/lib/waagent/
Azure Diagnostics for Linux

19. © 2015 Riverbed Technology. All rights reserved. 19
 Instrumentation for Azure
web and worker roles
 View data in Azure Portal
 Easy to use for downtime
alerting.
 Instrument via SDK,
recorded web scripts,
agents, et al.
 We use it for monitoring
http://appinternals.trafficmanag
er.net
AppInsights

20. © 2015 Riverbed Technology. All rights reserved. 20

21. © 2015 Riverbed Technology. All rights reserved. 21
 Data resolution is not
intended to be
comprehensive.
AppInsights
“We don’t store all results”

22. © 2015 Riverbed Technology. All rights reserved. 22
 Outages do occur.
AppInsights
https://blogs.msdn.microsoft.com/applicationinsights-status/

23. Thank You
© 2015 Riverbed Technology. All rights reserved. 23
AppInsights Demo
Click here to start demo

24. © 2015 Riverbed Technology. All rights reserved. 24
Architecting
APM for Azure IaaS

25. © 2015 Riverbed Technology. All rights reserved. 25
Instrumenting SharePoint Farms in Azure IaaS
DEMO

26. © 2015 Riverbed Technology. All rights reserved. 26
 Automation and Orchestration
– Can you ensure APM agents are installed and configured automatically as
IaaS VMs are dynamically provisioned and destroyed?
 Scaling APM management
– Most APM solutions have limits on how much data they can handle.
Challenges for APM in IaaS

27. © 2015 Riverbed Technology. All rights reserved. 27
Architecting
APM for Azure PaaS

28. © 2015 Riverbed Technology. All rights reserved. 28
3rd Party APM vendors in Azure marketplace

29. © 2015 Riverbed Technology. All rights reserved. 29
Instrumenting with Startup Tasks and NuGet
APM for Azure PaaS
DEMOThis works for both Java and .NET cloud services!

30. © 2015 Riverbed Technology. All rights reserved. 30
 Scaling APM management
– Most APM solutions have limits on how much data they can handle.
Challenges for APM in PaaS

31. © 2015 Riverbed Technology. All rights reserved. 31
Architecting
APM for SaaS

32. © 2015 Riverbed Technology. All rights reserved. 32
 You can not install agents. So what’s left?
– JavaScript injection (via end-user customizations to the
SaaS app)
– Client-side network monitoring (e.g. Riverbed
AppResponse)
– Server-side network monitoring via cloud proxy (e.g.
Riverbed Steelhead)
 Scaling APM management
– Most APM solutions have limits on how much data they
can handle.
Challenges for APM in SaaS
In this presentation we
are focusing on APM.
We should discuss
Network Performance
Monitoring (NPM) later.

33. © 2015 Riverbed Technology. All rights reserved. 33
Embed JavaScript in
Master Pages
Network monitoring
Case Study: Instrument SharePoint Online (SPO)
Customized Master
Pages will not receive
updates from
Microsoft.
This is the best way to
embed JavaScript in
SPO sites.
Detailed on next slide.
Network appliances in
your corporate data
center can provide a
vantage point into
O365 performance.
For example, with
Riverbed Steelhead!
Embed JavaScript in
UserCustomAction

34. © 2015 Riverbed Technology. All rights reserved. 34
Embedding JS in Master Pages
SharePoint Online Instrumentation (Option #1)
1. Open Master Page Gallery
2. Check out seattle.master
3. Add JavaScript snippet in the
head tag
4. Upload and check in
seattle.master
DEMO

35. © 2015 Riverbed Technology. All rights reserved. 35
Embedding JS with SPUserCustomAction
SharePoint Online Instrumentation (Option #2)
1. Create a new “SharePoint Add-In” that is a provider hosted app
2. Get the list of UserCustomActions
3. Add a new custom action to that list, which invokes JavaScript on page load
See .NET API reference for SPUserCustomAction.ScriptBlock()
4. Update the set of UserCustomActions on the SharePoint site
DEMO
https://github.com/OfficeDev/PnP-Guidance/blob/master/articles/Customize-your-SharePoint-site-UI-by-using-JavaScript.md
https://github.com/OfficeDev/PnP/tree/master/Samples/Core.EmbedJavaScript
https://channel9.msdn.com/Blogs/Office-365-Dev/JavaScript-injection-in-SharePoint-Online-Office-365-Developer-Patterns-and-Practices
References:

36. © 2015 Riverbed Technology. All rights reserved. 36
Option #2 requires sideloading, or you’ll get this error:
Here’s how to enable sideloading of apps in SharePoint:
Install the SharePoint Online
Management Shell
Save this as Enable.ps1
Run Enable.ps1
Save this as Disable.ps1
Run Disable.ps1
http://pastebin.com/cLkJNnf3
http://pastebin.com/yqKNb1Mx

37. © 2015 Riverbed Technology. All rights reserved. 37
Detect Errors
Baseline
Performance
Track Users Security Audits
What does SaaS monitoring give you?
What is “normal”
performance?
Verify cloud SLAs
Who are the power
users?
What user accounts
are inactive?
Detect unauthorized
users
Geo locate intrusions
Do some pages
return errors?
What path lead to
errors?

38. © 2015 Riverbed Technology. All rights reserved. 38
Here’s what Riverbed AppInternals
can do with SaaS data..

39. © 2015 Riverbed Technology. All rights reserved. 39
http://appinternals.com

40. © 2015 Riverbed Technology. All rights reserved. 40
References
 Riverbed AppInternals - http://appinternals.com
 New Relic – http://newrelic.com
 Microsoft AppInsights – https://azure.microsoft.com/en-
us/documentation/articles/app-insights-get-started/

41. Thank You
© 2015 Riverbed Technology. All rights reserved. 41
Questions?