Clear ×

Kafka

71 repositories found

datazip-inc

datazip-inc

olake
Go AI analysis

OLake Go is a high-performance, open-source data ingestion engine that replicates data from databases (PostgreSQL, MySQL, MongoDB, Oracle, DB2,...

Production ready Niche use case
apache-iceberg cdc change-data-capture
1.4k
236
144
12/wk
active
7/10 Data
DataTalksClub

DataTalksClub

data-engineering-zoomcamp
Jupyter Notebook AI analysis

Data Engineering Zoomcamp is a free, structured 9-week educational course teaching data engineering fundamentals through hands-on pipeline building...

Beginner friendly
course data-engineering dbt
43.4k
8.6k
392/wk +45 yesterday
recent
8/10 Education
warpstreamlabs

warpstreamlabs

bento
Go AI analysis

Bento is a high-performance stream processor that connects various data sources and sinks (Kafka, cloud services, databases, etc.) with declarative...

Production ready Beginner friendly Niche use case
benthos bento connect
2k
203
138
11/wk
active
8/10 DevOps
openmeterio

openmeterio

openmeter
Go AI analysis

OpenMeter is an open-source real-time metering and billing platform designed specifically for AI, API, and DevOps monetization. It enables companies...

Production ready Niche use case
analytics billing clickhouse
2.1k
191
131
17/wk
active
8/10 Finance
svix

svix

svix-webhooks
Rust AI analysis

Svix is an enterprise-ready webhook service and dispatcher written in Rust that handles delivery, retries, security, and reliability for webhook...

Production ready
api background-jobs background-tasks
3.3k
258
65
13/wk
active
8/10 DevOps
strimzi

strimzi

strimzi-kafka-operator
Java AI analysis

Strimzi provides a Kubernetes operator for deploying and managing Apache Kafka clusters on Kubernetes and OpenShift, abstracting away the complexity...

Production ready Niche use case
data-stream data-streaming data-streams
5.9k
1.5k
150
18/wk
active
8/10 DevOps
arkflow-rs

arkflow-rs

arkflow
Rust AI analysis

ArkFlow is a high-performance Rust stream processing engine with integrated AI/ML capabilities, designed for real-time data processing, inference,...

Production ready Niche use case
ai arkflow datafusion
1.3k
46
32
7/wk
recent
7/10 Data
fede1024

fede1024

rust-rdkafka
Rust AI analysis

rust-rdkafka is a fully asynchronous, futures-based Apache Kafka client library for Rust that wraps librdkafka. It serves Rust developers building...

Production ready Single maintainer Niche use case
futures kafka kafka-client
2k
355
152
4/wk
recent
8/10 DevOps
debezium

debezium

debezium
Java AI analysis

Debezium is an open-source change data capture (CDC) platform that monitors databases and streams row-level changes as events to Apache Kafka. It...

Production ready Niche use case
apache-kafka cdc change-data-capture
12.9k
3k
159
26/wk
active
8/10 Data
confluentinc

confluentinc

examples
Java AI analysis

This is an official Confluent repository hosting curated demos and examples for Apache Kafka, Apache Flink, and Confluent Platform, showcasing event...

Production ready Beginner friendly
avro cdc cloud
2.1k
1.1k
109
4/wk
recent
7/10 Data
kafbat

kafbat

kafka-ui
TypeScript AI analysis

Kafbat UI is an open-source web dashboard for monitoring and managing Apache Kafka clusters, enabling operators to inspect brokers, topics, consumer...

Production ready Niche use case
apache-kafka big-data cluster-management
2.5k
351
276
13/wk
active
8/10 DevOps
memgraph

memgraph

memgraph
C++ AI analysis

Memgraph is a high-performance, in-memory graph database built in C++ that specializes in powering GraphRAG pipelines, AI memory systems, and agentic...

Production ready Niche use case
ai ai-agents ai-memory
4.2k
244
696
22/wk
active
8/10 AI & ML
ag2ai

ag2ai

faststream
Python AI analysis

FastStream is an asynchronous Python framework for building event-driven microservices with unified APIs across multiple message brokers (Kafka,...

Production ready Beginner friendly Niche use case
asyncapi asyncio distributed-systems
5.3k
360
102
25/wk
active
8/10 Dev Tools
akvorado

akvorado

akvorado
Go AI analysis

Akvorado is a specialized network flow collector, enricher, and visualization platform that ingests NetFlow/IPFIX and sFlow data, augments it with...

Production ready Niche use case
clickhouse collector ipfix
2.3k
170
37
10/wk
active
8/10 DevOps
robustmq

robustmq

robustmq
Rust AI analysis

RobustMQ is a unified messaging engine written in Rust that consolidates multiple messaging protocols (MQTT, Kafka, AMQP, NATS) into a single binary...

Niche use case
agent-communication ai ai-agnet
1.6k
235
111
7/wk
active
6/10 DevOps
flexprice

flexprice

flexprice
Go AI analysis

Flexprice is a usage-based and credit-based billing infrastructure platform designed for SaaS and AI-native companies. It provides real-time...

Production ready Niche use case
billing clickhouse events
3.6k
181
178
3/wk
active
7/10 Finance
twmb

twmb

franz-go
Go AI analysis

Franz-go is a pure Go Apache Kafka client library supporting versions 0.8.0 through 4.2+, offering comprehensive features including transactions,...

Production ready Niche use case
client go golang
2.9k
288
10
11/wk
active
9/10 DevOps
dotnetcore

dotnetcore

CAP
C# AI analysis

CAP is a .NET library that implements distributed transactions and event bus functionality for microservices, using the Outbox Pattern with local...

Production ready Niche use case
distributed-transactions eventbus kafka
7.1k
1.3k
8
3/wk
recent
8/10 DevOps
AutoMQ

AutoMQ

automq
Java AI analysis

AutoMQ is a diskless Kafka implementation that runs on S3, designed for cloud-native architectures seeking significant cost reduction and elastic...

Production ready Niche use case
automq diskless kafka
10.2k
731
75
37/wk
active
8/10 DevOps
apache

apache

kafka
Java AI analysis

Apache Kafka is a distributed event streaming platform designed for high-performance data pipelines, streaming analytics, and mission-critical...

Production ready Niche use case
java kafka scala
33.1k
15.3k
443
88/wk
active
9/10 DevOps
rsyslog

rsyslog

rsyslog
C AI analysis

Rsyslog is a high-performance, modular log ingestion and ETL engine written in C, capable of processing over one million messages per second with...

Production ready Single maintainer Niche use case
elasticsearch kafka logging
2.3k
726
252
6/wk
active
8/10 DevOps
redpanda-data

redpanda-data

redpanda
C++ AI analysis

Redpanda is a Kafka-compatible streaming data platform built in C++ without ZooKeeper or JVM dependencies, targeting developers who need lower...

Production ready Niche use case
containers cpp event-driven
12.3k
765
728
30/wk
active
8/10 Data
authorjapps

authorjapps

zerocode
Java AI analysis

Zerocode is a no-code automated testing framework for REST APIs, SOAP services, Kafka data streams, and databases using JSON/YAML configuration. It...

Production ready Beginner friendly Niche use case
api api-contract assertions
1k
445
133
2/wk
recent
7/10 Dev Tools
risingwavelabs

risingwavelabs

risingwave
Rust AI analysis

RisingWave is a SQL-based event streaming platform designed specifically for agentic AI and real-time applications that require continuously fresh,...

Production ready Niche use case
apache-iceberg data-engineering database
9.2k
790
1.6k
15/wk
active
8/10 Data
tchiotludo

tchiotludo

akhq
Java AI analysis

AKHQ is a web-based GUI for Apache Kafka that enables management of topics, consumer groups, schema registry, and Kafka Connect. It serves DevOps...

Production ready Single maintainer Niche use case
gui java kafka
3.8k
778
269
6/wk
active
7/10 DevOps
influxdata

influxdata

telegraf
Go AI analysis

Telegraf is a widely-adopted open-source agent for collecting, processing, and writing metrics, logs, and telemetry data across 300+ plugins. It...

Production ready
golang hacktoberfest influxdb
17.7k
5.8k
415
27/wk
active
8/10 DevOps
asyncapi

asyncapi

spec
JavaScript AI analysis

AsyncAPI is a specification for defining machine-readable asynchronous API contracts, similar to OpenAPI for REST APIs. It enables standardized...

Production ready Beginner friendly Niche use case
amqp asyncapi kafka
5.2k
383
42
7/wk
active
8/10 Dev Tools
apache

apache

camel
Java AI analysis

Apache Camel is a mature, enterprise-grade integration framework with 350+ connectors for databases, APIs, message brokers, and cloud services. It...

Production ready Niche use case
camel cloud-native data-transformation
6.3k
5.1k
18
8/wk
active
9/10 DevOps
PeerDB-io

PeerDB-io

peerdb
Go AI analysis

PeerDB is a specialized ETL tool that replicates data from PostgreSQL to data warehouses, message queues, and storage systems using CDC (Change Data...

Production ready Niche use case
bigquery cdc clickhouse
3.2k
198
193
4/wk
active
7/10 Data
ThreeDotsLabs

ThreeDotsLabs

watermill
Go AI analysis

Watermill is a Go library for building event-driven applications with support for multiple pub/sub transports (Kafka, RabbitMQ, NATS, HTTP,...

Production ready Beginner friendly Niche use case
cqrs event-driven event-sourcing
9.8k
499
80
12/wk
slow
8/10 Dev Tools

Showing 30 of 71