
Logs, OpenMetrics, and OpenTelemetry All-in-one
In this presentation, you will learn how to collect and aggregate Logs, Metrics, and Traces all together without losing control of your data by connecting different protocols at scale.
In this presentation, you will learn how to collect and aggregate Logs, Metrics, and Traces all together without losing control of your data by connecting different protocols at scale. Observability continues to be an interesting challenge, data collection for logs, metrics, and traces is an expensive operation and different approaches exist. In this talk we discuss why production ready OpenSearch Fluent Bit and Fluentd connectors are important to the project, and how these technologies work together to build an end to end log analytics solution. This hands-on session will give attendees a chance to both learn and try out the new capabilities while being guided by a Fluent maintainer and Prometheus community expert. In this session, Fluent Operator maintainers will elaborate on the key features of Fluent Operator as well as its design principles and architecture. This talk will describe our Fluent Bit monitoring story and it will briefly touch on some of the design choices but will primarily focus on how to monitor Fluent Bit in the production clusters. In this talk, we introduce a new feature in the in_tail input plugin, which uses group rules to rate limit log collection, and also we present a Log Flow Control framework that allows users to define and enforce log rate limit policies to control log loss predictably. The talk brings you into the community discussion on how to best optimize this event loop system and uncovers efforts spent on implementing the identified improvements. In this talk we will describe our journey from arbitrary log collection from edge devices to building an observability pipeline with FluentBit. In this session, we’ll take a look at how we modernised our infrastructure, and used the Fluent Node library to meet these challenges, eventually scaling up to 3.5 billion events per day. Closing Remarks with Anurag Gupta from Calyptia Fluentcon Europe
Presentations from the Calyptia Fluentcon conference May 16, 2022, co-located with Kubecon Europe.
Logs, OpenMetrics, and OpenTelemetry All-in-one
Logs, Prometheus and OpenTelemetry
Using the OpenSearch Project and Fluent
Getting started with Fluent Bit and Metrics with Prometheus
Fluent Operator | Intro
Monitoring Fluent Bit in Production
Data Flow Control in Cluster Logging Pipeline
Fluent Community Meeting
Demystify the Fluent Core Event Loop and Join in on its Optimization Efforts
Challenges of Logging on the Edge
Modernizing to the Fluent Stack: An Asana Story
Closing Remarks