Overview

This talk focuses on LinkedIn’s efforts and experiences to build up the observability pipelines with the help of open source standards and tools, specifically OTEL and Fluent Bit. Starting with tracing, we adopted the industry standard OpenTelemetry (OTEL), including both tracing schema and SDKs. This gave us a chance to leverage the already existing and supported tools and components. We also decided to introduce the transport agent to hide all the details of accessing the data backend from the application, therefore get the much needed multi-language support. (Not) surprisingly, Fluent Bit was chosen as our transportation agent, as opposed to the OTEL agent already provided in the OTEL SDK. This decision was made mainly based on the following considerations: – Kafka is our preferred and chosen centralized data collector – Log streaming is the required functionality. – Resource efficiency and performance. We also enhanced fluent-bit in the following ways: – OTEL transport agent with protobuf support – Metrics emission for Monitoring – Tag management and enhancement We’d be happy to contribute these enhancements back to the open source community in the future.

Presentation Deck

Download

Presenter

Chao Xu
Staff Software Engineer, Systems Infrastructure, LinkedIn

Chao Xu is a software engineer working at Service Infrastructure (SI) team of LinkedIn. With his recent focus on observability, he helped to build up the next generation trace/log data pipeline, which is expected to generate/stream many petabytes of daily data in order to provide the full insight and debuggability within the LinkedIn system. He also investigates the efficient data streaming solutions through collector/data plane and associated deployment models in different environments. Before that, Chao had contributed to multiple areas, including network, service discovery and load balancing improvements and innovations.