Full stack monitoring across apps & infrastructure with Azure Monitor

PROTECT
SECURE
MONITOR
CONFIGURE
GOVERN
Security Management
Threat Protection
Backup
Disaster Recovery
Policy Management
Cost Management
Configuration
Update Management
Automation
Scripting
Azure
Log Analytics
App, Infra & Network Monitoring
MIGRATE

Azure Monitor Documentation –
https://docs.microsoft.com/en-us/azure/azure-
monitor/ - new consolidated landing page for
LA/AppInsight
https://aka.ms/MonitoringDocs
Resources, Tutorials & What’s New –
https://aka.ms/AzMonOverview

https://aka.ms/Ignite2018/AzureMonitor
Azure Monitor E2E w/ Announcements
Azure Monitor for VMs/Containers & Log Analytics
Continuous Monitoring for DevOps
Network Monitoring
Telemetry & ITSM/SIEM Partners
Smart Alert Management

One Metrics. One Logs. One
Alerts.
Log Analytics query experience
integrated into Azure Portal
Integration into native Azure
resource blades
Configure Azure AD to send
audit & sign-up logs to Azure
Monitor
Ability to send Custom Metrics
Azure Monitor for resource
groups
Azure Monitor for VMs (health,
performance, and maps)
Multi-cluster health rollup view
for AKS
Distributed Tracing for Python/Go
in addition to .NET, Java &
Node.js apps
Java Local Forwarder, Micrometer
& Spring Boot support
Onboard VMs at scale via
Azure Policy
Secured monitoring inside
Virtual Networks
Store logs in firewall restricted
secured storage accounts
Monitor ER and store NSG
flow logs across subscriptions
Scale & SecurityData Driven InsightsUnified Monitoring

Advanced diagnostics and
analytics powered by machine
learning capabilities
Data Driven Insights
Rich ecosystem of popular
DevOps, issue management,
SIEM, and ITSM tools
Workflow Integrations
A common platform for
all metrics, logs and other
monitoring telemetry
Unified Monitoring
Metrics Log
Common Store
Full observability for your infra, app and network

Azure
Subscriptions
Security Center
Resource Manager
Service Health
Azure
Resources
Network Security Groups
Virtual Machines
Storage Accounts
Guest OS
(‘user space’)
Linux syslog
Windows Perf Counters
Application
User telemetry
Application logs
Azure Tenants Azure Active Directory
Signals Sources

Metrics
Logs
Application Container VM Monitoring
Solutions
Insights
Dashboards Views Power BI Workbooks
Visualize
Metrics Explorer Log Analytics
Analyze
Alerts Autoscale
Respond
Event Hubs Ingest &
Export APIs
Logic Apps
Integrate
Azure Monitor
Custom Sources
Application
Operating System
Azure Resources
Azure Subscription
Azure Tenant

Metrics
Logs
Azure Monitor
Custom Sources
Application
Operating System
Azure Resources
Azure Subscription
Azure Tenant
Logs & Metrics
emitted by Azure
Diag. Extensions + Agents
Windows + Linux Support
Workload Agnostic
Application Insights
SDK Driven
Multi-Language Support
For everything else

Unified Monitoring
Integration into native Azure
resource blades
One Metrics, One Logs, One Alerts
across Azure/on-prem resources
Ability to send custom metrics &
custom logs
Unified offering with App Insights &
Log Analytics as integrated features

Jump to Application Map or VM Map
Monitor health state of all resources
Drill down into failures or perf issues
See alerts firing across app & infra

Track E2E distributed transactions
(including for Python & Go) NEW!
Monitor apps in .NET, JS, Java, Node.js
or any language with OSS SDKs NEW!
Drill down to code-level with
Snapshot Debugging & Profiling
Visualize server/client connections
& dependencies with App Map
Understand end-user cohorts,
behavior & engagement for planning

Visualize service dependencies &
connection failures in Maps
Monitor single VMs or at scale
Onboard at scale using PowerShell
or Azure Policy
Troubleshoot perf issues like CPU,
memory, disk, and network
Identify & isolate host-level or
guest-level health problems

Understand cluster capacity needs
under average or heaviest loads
Monitor multi-cluster health &
node/pod status NEW!
Monitor containers on demand in
AKS with virtual nodes NEW!
Analyze Kubernetes event &
container logs for troubleshooting
View overall perf across nodes,
controllers and containers

Monitor ExpressRoute connectivity
to virtual networks and O365
Secure and audit your network with
Network Watcher Traffic Analytics
Monitor connectivity to LoB apps
with Service Connectivity Monitor
Discover and monitor ExpressRoute
circuits, across subscriptions

Advanced Queries with Log Analytics
Run queries for investigations,
statistics & root cause/trend analyses
Log Analytics advanced query
experience now in Azure Portal NEW!
Utilize ML algorithms for clustering
and anomaly detection
Central Analytics Platform across
Monitoring, Management, Security

Integration with Monitoring & SIEM Tools
Integrate your existing APM/Monitoring
solutions with Azure Monitor
Azure Monitor is best for Azure, and
provides both APM & SIEM capabilities
Route telemetry to your SIEM solutions
for analytics & security management
Open and extensible to continue using
your favorite tools & solutions

Partner Solutions for Apps & Workloads

 Rich insights for Azure Storage, SQL & Service Fabric Mesh
 RBAC per data type in Log Analytics
 Support for containers on demand for AKS with virtual nodes
 Support live logs for containers and Kubernetes events
 Smart detections and alerts with Dynamic Thresholds
 Expanded Azure support and interoperability for Distributed Tracing
 Exception tracking and custom metrics for Python and Go apps
 Auto enablement of app monitoring for App Services / Windows VMs (with WAD)
 Expanded region availability across public & sovereign clouds
 Latency improvements for Alerts, Logs & Metrics

Visibility – Get the Big Picture
Near Real-time Alerts & Notifications
Multi-Dimensional Metrics
Health & Availability Monitoring
Azure Dashboards
Insights – Find & Fix Problems
Composite Application & Service Maps
Distributed Transaction Tracing
Advanced Analytics with ML
Automated Actions & Remediations
Optimization – Build, Measure, Learn
Performance Optimization & Profiling
User Behaviour & Customer Insights
Impact Correlation
Integration with Dev/DevOps Tools

Gain visibility across workloads
Enable consistent control and
compliance
Respond faster to security threats
Ensure availability of apps and
data
Insight &
Analytics
Protection &
Recovery
Security &
Compliance
Automation
& Control
https://docs.microsoft.com/en-gb/azure/azure-monitor/azure-
monitor-rebrand#retirement-of-operations-management-suite-
brand

System
Center
System Center
2007
System Center
2012
System Center
2012 R2
System Center
2016/2019…
Operations Management Suite / Log Analytics / Security Center
Alerts
~30sec
to 5mins

Full Stack Monitoring & Analytics across Apps and Infra
Application Insights
Scenario Specific Monitoring – Customized Data Ingestion & Diagnostics
Log Analytics
Service Map Container Health
…Network Performance Monitor
Monitoring Fundamentals – Available out of the box with Azure Platform
Activity LogsDiagnostic Logs Service HealthMetrics
Dashboards Alerts Action Groups Autoscale
Unified pricing model
Only pay what you use
Data ingestion per GB

Private or hosted third-party cloud,
Rackspace, etc.
WINDOWS
WINDOWS
WINDOWS
WINDOWS
Public cloud
Azure or AWS
Simplified guest and workload management, both on-premises and in the cloud
On-premises with System Center or Direct Agent
WINDOWS
HYPER-V
WINDOWS
VMWare
WINDOWS
Log Analytics /
ASC

https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-manage-access
Today, a workspace provides:
• A geographic location for data storage
• Granularity for billing
• Data isolation
• Scope for configuration
Based on the preceding characteristics, you may want to create multiple workspaces if:
• You are a global company and you need data stored in specific regions for data sovereignty or compliance reasons.
• You are using Azure and you want to avoid outbound data transfer charges by having a workspace in the same
region as the Azure resources it manages.
• You want to allocate charges to different departments or business groups based on their usage. When you create a
workspace for each department or business group, your Azure bill and usage statement shows the charges for each
workspace separately.
• You are a managed service provider and need to keep the Log Analytics data for each customer you manage
isolated from other customer’s data.
• You manage multiple customers and you want each customer / department / business group to see their own data
but not the data for others.
My addition: To set different retentions for different data types i.e. Security vs. Events

Guidance:
Please try and create as few large workspaces as you can!
Have a playground workspace as well for testing and to
monitor your actual usage and costs!
An average Azure VM ingests ~1 GB to 3 GB of data per month.
Log Analytics + Azure Security Center = all data in one place,
less joins or cross workspace issues
E.g.
Today
One Workspace in
a region, with all
data and all users
You may need secondary (or more) workspaces
because:
1. Geographical concerns
2. Compliance
3. Latency – esp if you are a global org
Note: data that is sent from other regions
includes outbound data transfer charges.
4. RBAC (being addresses – in preview stage)
https://docs.microsoft.com/en-
us/azure/log-analytics/log-analytics-
manage-access#determine-the-
number-of-workspaces-you-need

US
Workspace
EU
Workspace
Asia
Workspace

•Windows or Linux computers and virtual machines. You install the Microsoft Monitoring Agent on Windows and Linux
•Azure services. Log Analytics collects telemetry from Azure Diagnostics and Azure Monitoring into the repository
•Data Collector API. Log Analytics has a REST API for populating data from any client.

Note: near-real time alerting is listed here: https://docs.microsoft.com/en-us/azure/monitoring-
and-diagnostics/monitoring-near-real-time-metric-alerts

Please use the Azure Portal to access the product not the
Legacy OMS portal.
Simply from Portal.Azure.Com, select your Log Analytics
workspace – then press WORKSPACE SUMMARY or Log
Search rather than OMS Portal.
The old portal is being phased out and many features are
no longer available there.

https://azure.microsoft.com/en-gb/blog/introducing-the-redesigned-security-
center-overview-dashboard/

Creating dashboards with
content spanning Log
Analytics, Application
Insights and Azure
http://blogs.catapultsyste
ms.com/…/creating-
dashboards-wit…/ and
this one

Application Insights is an extensible Application
Performance Management (APM) service for web
developers on multiple platforms. Use it to monitor your
live web application. It will automatically detect
performance anomalies. It includes powerful analytics
tools to help you diagnose issues and to understand what
users actually do with your app. It's designed to help you
continuously improve performance and usability. It works
for apps on a wide variety of platforms including .NET,
Node.js and J2EE, hosted on-premises or in the cloud. It
integrates with your DevOps process, and has connection
points to a variety of development tools. It can monitor
and analyze telemetry from mobile apps by integrating
with Visual Studio App Center and HockeyApp.
https://docs.microsoft.com/en-us/azure/application-
insights/app-insights-overview
Uses a workspace like Log Analytics – You can query App
Insights from Log Analytics e.g.

Query
//per computer
union withsource = tt * | where _IsBillable == true | summarize Bytes=sum(_BilledSize) by Computer |
sort by Bytes nulls last
Links:
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-usage?toc=/azure/azure-
monitor/toc.json#troubleshooting-why-usage-is-higher-than-expected
October 2018 updated SLA
https://azure.microsoft.com/en-us/support/legal/sla/log-analytics/v1_3/
New Ingestion time defined Nov 2018
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-ingestion-
time?fbclid=IwAR08sXlFCyOcnTvlwU4QE12P38CxuIFwWyOWhHwlGfHlyxE2LlSisaaQtk4

https://azure.microsoft.com/en-us/blog/introducing-a-new-way-to-purchase-azure-monitoring-services/
Should I move to the new April 1st 2018 licence model – use the cost estimator ?Should
https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-usage-and-estimated-cos
1. Consistent pay-as-you-go pricing
We are adopting a simple “pay-as-you-go” model across the complete portfolio of monitoring services.
You have full control and transparency, so you pay for only what you use.
2. Consistent per gigabyte (GB) metering for data ingestion
We are changing the pricing model for data ingestion from “per node” to “per GB”. Customers told us
that the value in monitoring came from the amount of data received and the insight built on top of that,
rather than the number of nodes. In addition, this new model works best for the future of containers and
microservices where the definition of a node is less clear. “Per GB” data ingestion is the new basis for
pricing across application, infrastructure, and networking monitoring.
Learn more about the new pricing model by visiting the updated pricing calculator or the individual product
pricing pages for Log Analytics, Network Watcher, Azure Monitor, and Application Insights.

We support the CEF format of logs and can accept those, OMS Security supports
collection of logs using CEF over Syslogs and Cisco ASA logs, Arcsight etc...
What is CEF?
Common Event Format (CEF) is an industry standard format on top of Syslog
messages, used by many security vendors to allow event interoperability
among different platforms. OMS Security and Audit Solution support data
ingestion using CEF, which enables you to connect your security products
with OMS Security.

This is a performance expensive operation – its an exception. E.g. for GDPR “forget
me” requests.
https://docs.microsoft.com/en-us/rest/api/loganalytics/workspace/purge
Top Tip: Be VERY careful with this, it’s a one time
action – if you have support maybe involve
them

Activity
log
https://docs.microsoft.com/en-us/azure/security-center/security-center-
export-data-to-siem

http://www.microsoft.com/handsonlabs/
Language Reference
https://portal.loganalytics.io/demo#/disc
over/query/main

Useful Links
www.microsoft.com/systemcenter
http://blogs.technet.microsoft.com/hybridcloud
http://blogs.msdn.microsoft.com/ukhybridcloud
http://aka.ms/OMSSecBlog
https://blogs.technet.microsoft.com/msoms/
https://feedback.azure.com/forums/267889-log-analytics?query=tag
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-service-providers
https://azure.microsoft.com/en-us/blog/announcing-the-new-and-improved-azure-log-analytics/preview/
https://techcommunity.microsoft.com/t5/Azure-Log-Analytics/bd-p/AzureLogAnalytics
https://portal.loganalytics.io/demo#/discover/home
https://www.facebook.com/groups/MicrosoftOMS/
https://github.com/MicrosoftDocs/LogAnalyticsExamples/tree/master/log-analytics
https://azure.microsoft.com/en-us/updates
https://www.pluralsight.com/courses/kusto-query-language-kql-from-scratch
https://azurenotes.tech/ServiceNotes?page=1&currentFilter=Log%20Analytics

Getting started with queries
 This is a series of posts to get you started with the query language
 Azure Log Analytics: Queries, the basics explained – Part 1

Kusto Query Language (KQL) course
 Free course on the Query language (should be free mid July, in the meantime you will need
Pluralsight access)
https://www.pluralsight.com/courses/kusto-query-
language-kql-from-scratch

Azure Advisors – if you have an NDA
with us?
http://aka.ms/azureadvisors
http://aka.ms/azureadvisorsaddcontact
http://aka.ms/joinadvisors

FAQ
Most Common Questions:
To deploy agents, do we need to have target computer objects in Azure?
OMS agents can be installed 3 ways. Directly from OMS portal, SCOM agent can be used as well to upload data to OMS via Sys Center Operations Manager or by simply enabling OMS agent as
extension in Azure VMs.
Connect Windows computers to the Log Analytics service in Azure
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-windows-agents
Connect Azure virtual machines to Log Analytics with a Log Analytics agent
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-vm-extension
Connect Operations Manager to Log Analytics
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-om-agents
How do we deploy the agent, via OMS portal, or can we do it with SCCM?
You cannot push agent via OMS portal but in Azure portal via one click (Connect button) you can install agent on VM and yes via SCCM you can push OMS agent as well.
Do we configure the agent to pull the data we want to or it just collects all by default and we can then filter it, or make required report?
For this I encourage to understand the concept of Data sources in Log Analytics. Lot of data is captured out of the box but certain data types can be captured on demand or tuned on or off like
events, perf counters etc.
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-sources
What is the limitation in trial, can you deploy agent to 50 or 5000 machines?
An OMS workspace on Free Tier won’t allow more than 500 MB data upload from all
machines. There is no number of agents limit but total data allowed to upload and data
retention (7 days on Free tier) limit.
And in the end, what is the estimated cost of OMS under subscription, is it calculated
per device?
On Licensing https://www.microsoft.com/en-cy/cloud-platform/operations-management-
suite-pricing
ADDITIONAL INFORMATION:
Computer groups in Log Analytics log searches
https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-computer-groups
Computer groups in OMS
https://blogs.technet.microsoft.com/msoms/2016/04/04/computer-groups-in-oms/
How to verify that your Microsoft OMS agents are working properly
https://blogs.technet.microsoft.com/omsblog/2016/01/05/how-to-verify-that-your-
microsoft-oms-agents-are-working-properly/
How to troubleshoot Operations Management Suite onboarding issues
https://support.microsoft.com/en-us/kb/3126513

Full stack monitoring across apps & infrastructure with Azure Monitor

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Full stack monitoring across apps & infrastructure with Azure Monitor

Semelhante a Full stack monitoring across apps & infrastructure with Azure Monitor (20)

Último

Último (20)

Full stack monitoring across apps & infrastructure with Azure Monitor

Notas do Editor