The document discusses Grafana Labs' open source observability tools including Prometheus, Grafana, Loki, and Tempo. It provides an overview of each tool's capabilities and Grafana Labs' contributions to them. It also describes how Loki provides highly scalable and cost-effective log aggregation through its storage model and query processing.
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
OSMC 2022 | The Power of Metrics, Logs & Traces with Open Source by Emil-Andreas Siemes
1. The Power of Metrics, Logs &
Traces with Open Source
2. ● Loves running
● Baking sourdough bread
https: //www.linkedin.com/in/emil-andreas-siemes-a793926/
Emil Siemes
Principal Solutions Engineer
3. 1000+ Employees across
40+ countries
(50+ in Germany)
1M+
Instances across
LGTM Cloud and
OSS
13M+ Users across OSS
and Cloud Free tier
Founded in 2014
4. We help our customers Build and Operate
resilient systems,
to provide reliable services to their
customers.
6. Prometheus
Largest contributor - employs
44% of Prometheus
maintainers
Open source is at the of what we do
GRAFANA
Vizualize, Correlate & Alert
on Multi-sourced Data
Grafana LOKI
Highly Scalable Log
Aggregation
Grafana MIMIR
Scalable & Performant
backend for all
time-series metrics
Graphite
Leading contributor on
Graphite
Grafana TEMPO
Highly Scalable &
Cost Effective Tracing
OpenTelemetry
Large contributor - employs
maintainters; first class
support in Grafana
K6
Modern load testing
built for developers
Grafana On-Call
Oncall management
system with simple
workflow integrated with
your telemetry
7. Prometheus
Largest contributor - employs
44% of Prometheus
maintainers
Open source is at the of what we do
GRAFANA
Vizualize, Correlate & Alert
on Multi-sourced Data
Grafana LOKI
Highly Scalable Log
Aggregation
Grafana MIMIR
Scalable & Performant
backend for all
time-series metrics
Graphite
Leading contributor on
Graphite
Grafana TEMPO
Highly Scalable &
Cost Effective Tracing
OpenTelemetry
Large contributor - employs
maintainters; first class
support in Grafana
K6
Modern load testing
built for developers
Grafana On-Call
Oncall management
system with simple
workflow integrated with
your telemetry
8. Deployment Flexibility
Open-source | Enterprise | Grafana Cloud
Big tent
Flexibility and openness without limits to integrate 3rd
party data
Observability by open source
A modern, complete and highly modulable platform fully
built on OSS
A Unique Philosophy : Regain Control Over Your
Observability Strategy
9. LGTM: A prometheus inspired O11y stack
Tempo
Backend compatible
with OpenTelemtry,
inexpensive at scale,
100% sampling is
possible
Loki
Low TCO, Highly
scalable and easy to
operate logging
system
Mimir
Highly scalable, highly
available multi-tenant,
durable storage
drop-in replacement
for Prometheus
10. Correlation by design to reduce MTTR
Exemplars
Labels & service discovery
{service="foo", region="eu-west",
instance="node-123"}
Auto-generated
metrics
Logs for trace
TraceID in logs
Logs to metrics
extraction
S3
13. An open source web SDK for
frontend application observability
14.
15. Frontend Application Observability
in Grafana Cloud
Fully managed offering for capturing, visualizing,
and analyzing frontend application telemetry data within
Grafana Cloud, powered by Grafana Faro
PRIVATE BETA available today for Grafana Cloud customers
16. Getting started with Grafana Faro is super simple
Install the Grafana Faro Web SDK:
Import it into your app:
npm install @grafana/faro-web-sdk
import { initializeFaro } from '@grafana/faro-web-sdk'
initializeFaro({
url: '<http://my-endpoint>',
apiKey: '<api_key>',
app: { name: 'my-app' },
});
23. 2019-12-11T10:01:02.123456789Z {app=”nginx”, env=”dev”}
Timestamp
with nanosecond precision Content log line
Labels/Selectors
key-value pairs
indexed unindexed
GET /about 1034 Debug “page not found”
Object
Storage
Object
Storage
Aka “the index” Aka “chunks”
The Loki storage model
24. ● Log any and all formats
● Smaller indexes
● Cheaper to run
● Fast queries
● Cut and slice your logs in dynamic ways
Grafana Loki
Query time processing
VS
● Decide on log formats
● Larger indexes
● More expensive to run
● Faster queries
● Restricted to format chosen at ingestion time
Traditional solutions
Ingest time processing
The better tradeoff
25. 10TB 200MB
Log Data Index
Think of it more like a table of contents than an index.
Loki does not index the contents of the logs. Instead, entries are
grouped into streams and indexed with Prometheus-style labels.
Efficient log storage
26. 1PB 80TB
Raw Logs Label match Brute force search -
heavily parallelized
Time frame
1TB 120GB+/s
Fast queries
Think of it as massively distributed grep.
27. Per Second A day A month Active tenants
700MB 43TB 1.25PB 4500
170Cores 1TB RAM
27
Large cluster in Grafana Cloud
28. {cluster=”us-central1”, job=~”dev/loki-.*”} |= “trace_id=7ca877dbda” |~ "SeriesStore.*"
Label matchers
● = contains string.
● != does not contain string.
● =~ matches regular expression.
● !~ does not match regular expression.
Filter expressions
● |= contains string.
● != does not contain string.
● |~ matches regular expression.
● !~ does not match regular expression.
LogQL: Log queries
Parser expressions
Formatter expressions
30. Grafana Agent
Promtail
● Targets discovery for Kubernetes, Syslog, files and more
● Automatically attach labels to your log lines
● Advanced pipeline mechanism for parsing, transforming and
filtering your logs
● Build and expose custom metrics from your logs data
But Loki is open.
logstash Lambda
Loki makes log collection easy
31. Single Binary
- Testing
- Small installations
without HA or
horizontal scaling
needs
SSD or
Microservices
- Horizontal scalability
- Separate Read/Write
paths
- Large installations
Grafana Cloud
● Batteries included:
○ Multi-tenant
○ Automatic scaling
and upgrades
○ RBAC
○ Per tenant QoS
Enterprise Logs
● Self hosted
● Supported by Grafana
Labs
● Security out of the box
● Part of the GES stack
31
How to run Loki