Calyptia and OpenSearch Partner to Build First-Party Connectors to Fluent Bit and Fluentd

New connectors for OpenSearch allow users to leverage free, open-source log and metric agents, Fluent Bit and Fluentd, to ingest data quickly and reliably

Today we are proud to announce that Calyptia, the creators and maintainers of the fast, lightweight, highly scalable open-source logging and metrics processors, Fluent Bit and Fluentd, is partnering with the OpenSearch project team to bring first-party connectors for OpenSearch to Fluent Bit and Fluentd.

OpenSearch is a community-driven open-source search and analytics suite derived from Elasticsearch and Kibana that helps with use cases such as application search, log analytics, and more.  Fluentd and Fluent Bit are part of a Cloud Native Computing Foundation (CNCF) graduated open-source project that enables practitioners of observability, security, and DevOps to process and route data from multiple sources to multiple destinations at high scale in complex, dynamic environments.

With the new connectors to Fluent Bit and Fluentd, OpenSearch users will now have a reliable and straightforward way to ingest data to OpenSearch from various data sources – including Kubernetes, network devices, application logs, and log files. Fluent Bit and Fluentd users will have a first-party plugin for OpenSearch to migrate, add, route, or filter their application data.

“Since the launch of OpenSearch, we have seen consistently increasing demand for production-ready OpenSearch connectors for Fluent Bit and Fluentd,” explained Calyptia co-founder and Fluent Bit Creator, Eduardo Silva. “We are super excited to be partnering with the OpenSearch project team to build these connectors and provide an end-to-end, vendor-neutral, and open source solution for collecting, parsing, storing, and analyzing log and metric data. It’s a big step forward towards our objective to enable First Mile Observability for everyone, connecting any data source with any back-end destination, and enabling users to get immediate insight into their systems.”

Following both projects’ principles for development, all new plugins will be fully open sourced under an Apache 2.0 license and available for users to consume, leverage, and contribute.

“The OpenSearch project is excited to partner with Calyptia on OpenSearch connectors for Fluent Bit and Fluentd,” stated Jules Graybill, Director, OpenSearch at Amazon Web Services (AWS). “OpenSearch is about working with the community to build great software that solves real problems. When used together, Fluent Bit, Fluentd, and OpenSearch make it easy to ingest, secure, search, aggregate, view, and analyze data at scale across the entire IT ecosystem.”

What’s Next?

The connectors to OpenSearch for Fluent Bit and Fluentd will be finalized in the coming weeks and available for users in upcoming releases. Additionally, we welcome both communities’ input, discussions, and contributions to build more integrations between these two projects.

How to Get Involved

There are many ways to get involved with the effort. We invite you to join OpenSearch community meetings or Fluent community meetings, which include updates and Q&A. Of course, you can also always open an issue on Fluent Bit GitHub, or join us in the Fluent Slack community to keep the conversation going.