1. Infrastructure monitoring made easy
From ingest to insight
Pablo Musa
Curriculum Architect | Elastic

2. Evolving Architectures ~↑ Monitoring Complexity
Hardware & software trends
are evolving in tandem
Higher resource utilization
increases monitoring complexity
• Orchestration/Hypervisor
• Dynamic/ephemeral jobs
• You can no longer "point" to
where that job lives
Shift to cloud-native yields
maintainable code, with costs
• Traditional licensing models don't scale
as well as your applications
• Hurdles with autoscaling
Monitoring Complexity

3. 3
Requirements for a
monitoring solution

4. Applications
VMs/Containers
Other DBs,
Services &
Middleware
Orchestration InfrastructureAPM
Metrics
Logs
Uptime
Uptime
APM Metrics
APM Logs
APM
APM
Metrics
Logs
Uptime
Metrics
Logs
Uptime
APM

5. Core features and functionality
Needs from a monitoring solution
• Support the full stack
• Easily ingest from new sources
• Monitor dynamic ecosystems
• Ability to interact with your data
– Aggregations and visualizations
– Different views based on who is
looking
• Rich and flexible alerting
• Long term, reliable storage
• Bonus points for full Observability

6. 6
Adding logs and metrics
to Elastic Observability
Multiple ingest paths
for observability streams

8. Instructions
right in Kibana
Several integrations
and the list is growing
• Download and install Metricbeat
and/or Filebeat
• Edit the configuration for destination
• Enable and configure the module
• Start the beats
• Explore!

9. Elastic Ingest Manager
Centralized ingest and configuration
• Deploy Elastic Agent
• Choose the integration type
• Register and configure the data source
• Specify the data you want to collect
• Explore!

10. Use your existing shippers
Core features and functionality
Your App
Prometheus
Exporter
Your App
Prometheus
Exporter
Metricbeat +
Elasticsearch
Prometheus
Server
Metricbeat +
Elasticsearch Azure Monitor

11. Autodiscover
Automatically monitor
new containers
• Perfect for dynamic ecosystems
• Automatically picks up new instances
• Works with K8s, Docker, AWS, etc.
• Hints based auto-discovery for K8s
• Full context backed by
Elastic Common Schema

12. Elastic Common Schema
Established, predictable fields

13. Text & Time Series Combined
• Inverted index + columnar store
• Powerful aggregations framework
• Optimized numeric field types (BKD
• Fast response even for
high-cardinality queries
• ILM & Data Rollups
• With all of the benefits of
the Elastic Stack
Powerful data store

14. 14
The power of Elastic in your
observability initiative
Using your data to make
observability actionable

15. Out-of-the-box visibility
• Ship with most integrations
• Mix and match for your needs
• Leverage Kibana drilldowns for
custom navigation paths
• Of course, dedicated Metrics
and Logs apps
Dashboards & Visualizations

16. Metrics App
Birds-eye view or drill down

17. tail -f across your infrastructure ++
Logs App

18. Automatically detect and alert
• Many types of alerts
• Prefiltering based on context
• Multiple facets per alert
– CPU and Memory
– Network TX and RX
• Automatically split alerts on chosen
field (per container/pod/host)
• Deviations in logging rates
Integrated Alerting

19. Anomaly Detection
Automatically detect and alert
• Automate anomaly detection at
scale
and across disparate data sources
• Find patterns in your logs
• Automatically call out anomalies
and outliers

20. The power of the Stack
Use the Stack, Luke
• Flexible, composite visualizations
with Lens and TSVB
• Pixel-perfect infographics with
Canvas
• Multiple query options including
PromQL
• Data analytics based on ML
• Powerful Security

21. 21
Tying it all together
Elastic Observability provides everything
that you need for infrastructure monitoring

22. Full Stack Support
Modules & Integrations
• Hosts, VMs, Containers
• Applications and Services
• Databases
• Message Queues
• Network / Audit
• And more

23. Easily add new sources
Out-of-the-box visibility
• Multiple out-of-the-box integrations
• Stackdriver (GCOS
• Azure Monitor
• CloudWatch
• Statsd
• Prometheus & OpenMetrics
• Just another index
– Business KPIs
– Sales vs error rates

24. Interact and interrogate your data
• Multiple perspectives, geared for
different users
• Flexible metrics exploration
• Integration with other observability data
streams
• Quickly create visualizations
• Flexible alerting...
Interactive, immersive UI

25. From multiple levels of detail
• Context-aware smart filtering
• Multiple detection criteria
dimensions
• Rich action options
• Built into your workflows
Flexible Alerting and ML

26. Leverage built-in data management
• Index lifecycle management
• Configurable redundancy
• Data rollups
• Highly available, distributed by nature
Long term, reliable storage

27. Needs from a monitoring solution
Core features and functionality
• Support the full stack
• Easily ingest from new sources
• Monitor dynamic ecosystems
• Ability to interact with your data
• Rich and flexible alerting
• Long term, reliable storage
• Bonus points for full Observability

28. Thank You