Modern cloud-native applications are incredibly complex systems. Keeping the systems healthy and meeting SLAs for our customers is crucial for long-term success. In this session, we will dive into the three pillars of observability - metrics, logs, tracing - the foundation of successful troubleshooting in distributed systems. You'll learn the gotchas and pitfalls of rolling out the OpenTelemetry stack on Kubernetes to effectively collect all your signals without worrying about a vendor lock in. Additionally we will replace parts of the Prometheus stack to scrape metrics with OpenTelemetry collector and operator.
39. 39
● For metrics and traces OpenTelemetry takes the approach of a
clean-sheet design, specifies a new API and provides full
implementations of this API in multiple languages.
● Our approach with logs is somewhat different. For OpenTelemetry to be
successful in logging space we need to support existing legacy of logs
and logging libraries, while offering improvements and better integration
with the rest of observability world where possible.
OpenTelemetry logs
40. 40
OpenTelemetry Operator - collecting logs
● File log receiver
○ Available in the “contrib” docker image
○ Example config - opentelemetry-collector-contrib/otel-collector-config.yml
○ Includes: /var/log/pods/*/*/*.log
○ Set of operators to parse data: json_parser, regex_parser, move…
○ Collectors resource attributes: namespace, pod name/UID/restarts, container name
● Fluentforwardreceiver - receive data from
46. ● OpenTelemetry Operator
○ Auto-instrumentation for Webservers
● Profiling vision see 0212-profiling-vision.md
● OpAMP: Open Agent Management Protocol see open-telemetry/opamp-spec
46
What is next in OpenTelemetry?