Skip to content

Kafka architecture principles

Apache Kafka is designed as a distributed, partitioned, replicated commit log service, primarily used to build real-time data pipelines and streaming applications.^[600-developer-kafka-kafka.md]

Core Principles

Kafka's architecture is built on several fundamental design choices:

  • Distributed Commit Log: At its core, Kafka treats data as a structured, ordered commit log.^[600-developer-kafka-kafka.md]
  • Partitioning: Data is distributed across multiple servers (brokers) via partitions to allow for horizontal scaling and high throughput.^[600-developer-kafka-kafka.md]
  • Replication: Data is replicated across multiple nodes to provide fault tolerance and high availability.^[600-developer-kafka-kafka.md]

Sources

^[600-developer-kafka-kafka.md]