Friday, 17 December 2021

kafka

1. offset explorer

2. topics, partitions and offset

  • data is kept for a limit time (default 1 week)
  • data is randomly assigned to partition unless key is provided

3. brokers and topics
  • brokers are servers, if you connect to one broker, you connect to all
  • a broker store topic.partition, e.g. broker1 can store topic1.partition1 and topic2.partition3
  • topic has replication factor > 1, meaning redundant in storage


4. consumer group

  • each consumer in group read 1 or more partition, and exclusively


reference

1. apache kafka in 5 minutes