Learn All About Kafka Consumers. Learn about Kafka consumers and how to… | by Ashutosh Narang | Feb, 2022

Study Kafka shoppers and the best way to reliably eat knowledge from Kafka matters

Ashutosh Narang
Kafka Producer — Dealer — Shopper
Eat Course of and Retailer
N Partitions 1 Shopper
N Partitions M Customers (N > M > 1)
N Partitions M Customers (N == M)
N Partitions M Customers (N < M)
N Partitions M Customers (N < M) 2 Shopper Teams
Partition Rebalancing GIF

Create a KafkaConsumer Occasion

Properties props = new Properties();props.put("bootstrap.servers", "broker1:9092,broker2:9092");
props.put("group.id", "ConsumerGroup1");
props.put("key.deserializer", StringDeserializer");
props.put("worth.deserializer", StringDeserializer");
KafkaConsumer<String, String> shopper = new KafkaConsumer<String,
String>(props);

Subscribe to a Subject

shopper.subscribe(Collections.singletonList("topic1"));

Write the ballot loop

strive 
whereas (true)
ConsumerRecords<String, String> data = shopper.ballot(100);
for (ConsumerRecord<String, String> report : data)
System.out.println(report.toString())


lastly
shopper.shut();

State of affairs-1: Completely happy State

Dedicated Offset == Offset of Final Message Processed by the Shopper

State of affairs-2: Knowledge Loss

Dedicated Offset > Offset of Final Message Processed by the Shopper

State of affairs-3: Double Processing

Dedicated Offset < Offset of Final Message Processed by the Shopper

Automated Commit

Commit Present Offset

Synchronous Commit

whereas (true) 
ConsumerRecords<String, String> data = shopper.ballot(100);
for (ConsumerRecord<String, String> report : data)
System.out.printf("worth = %s", report.worth());

shopper.commitSync();
strive
shopper.commitSync();
catch (CommitFailedException e)
log.error("commit failed", e)

whereas (true) 
ConsumerRecords<String, String> data = shopper.ballot(100);
for (ConsumerRecord<String, String> report : data)
System.out.printf("worth = %s", report.worth());

shopper.commitAsync();

More Posts