Kafka 0.11 how to reset offsets

By default, –reset-offsets just prints the result of the operation. To actually perform the operation you need to add –execute to your command: kafka-consumer-groups.bat –bootstrap-server kafka-host:9092 –group my-group –reset-offsets –to-earliest –all-topics –execute

How to change start offset for topic?

Since kafka 0.11.0.0 you can use the script kafka-consumer-groups.sh Example from this answer kafka-consumer-groups.sh –bootstrap-server kafka-host:9092 –group my-group –reset-offsets –to-earliest –all-topics –execute Other options listed in the KIP-122: Add Reset Consumer Group Offsets tooling .———————-.———————————————–.———————————————————————————————————————————————-. | Scenario | Arguments | Example | :———————-+———————————————–+———————————————————————————————————————————————-: | Reset to Datetime | –to-datetime YYYY-MM-DDTHH:mm:SS.sss±hh:mm | Reset to first offset … Read more

Use Confluent Hub without Confluent Platform installation

Specify –component-dir and –worker-configs explicitly You’ll have to pass both options to include the Homebrew location of Kafka Connect’s plugin.path setting (where Confluent Hub would install to) and your connect-distributed.properties (so that the plugin path can be setup if it isn’t already) e.g. (on a Mac, after brew install kafka) confluent-hub install <name> \ –component-dir … Read more

Delete topic in Kafka 0.8.1.1

Deleting topic isn’t always working in 0.8.1.1 Deletion should be working in the next release, 0.8.2 kafka-topics.sh –delete –zookeeper localhost:2181 –topic your_topic_name Topic your_topic_name is marked for deletion. Note: This will have no impact if delete.topic.enable is not set to true. You may also pass in a bootstrap server instead of zookeeper: kafka-topics.sh –bootstrap-server kafka:9092 … Read more

How to change the number of replicas of a Kafka topic?

To increase the number of replicas for a given topic you have to: 1. Specify the extra replicas in a custom reassignment json file For example, you could create increase-replication-factor.json and put this content in it: {“version”:1, “partitions”:[ {“topic”:”signals”,”partition”:0,”replicas”:[0,1,2]}, {“topic”:”signals”,”partition”:1,”replicas”:[0,1,2]}, {“topic”:”signals”,”partition”:2,”replicas”:[0,1,2]} ]} 2. Use the file with the –execute option of the kafka-reassign-partitions tool [or … Read more

Kafka how to read from __consumer_offsets topic

I came across this question when trying to also consume from the __consumer_offsets topic. I managed to figure it out for different Kafka versions and thought I’d share what I’d found For Kafka 0.8.2.x Note: This uses Zookeeper connection #Create consumer config echo “exclude.internal.topics=false” > /tmp/consumer.config #Consume all offsets ./kafka-console-consumer.sh –consumer.config /tmp/consumer.config \ –formatter “kafka.server.OffsetManager\$OffsetsMessageFormatter” … Read more