How to change the number of replicas of a Kafka topic?

by

To increase the number of replicas for a given topic you have to:

1. Specify the extra replicas in a custom reassignment json file

For example, you could create increase-replication-factor.json and put this content in it:

{"version":1,
  "partitions":[
     {"topic":"signals","partition":0,"replicas":[0,1,2]},
     {"topic":"signals","partition":1,"replicas":[0,1,2]},
     {"topic":"signals","partition":2,"replicas":[0,1,2]}
]}

2. Use the file with the –execute option of the kafka-reassign-partitions tool

[or kafka-reassign-partitions.sh – depending on the kafka package]

For example:

$ kafka-reassign-partitions --zookeeper localhost:2181 --reassignment-json-file increase-replication-factor.json --execute

3. Verify the replication factor with the kafka-topics tool

[or kafka-topics.sh – depending on the kafka package]

 $ kafka-topics --zookeeper localhost:2181 --topic signals --describe

Topic:signals   PartitionCount:3    ReplicationFactor:3 Configs:retention.ms=1000000000
Topic: signals  Partition: 0    Leader: 2   Replicas: 0,1,2 Isr: 2,0,1
Topic: signals  Partition: 1    Leader: 2   Replicas: 0,1,2 Isr: 2,0,1
Topic: signals  Partition: 2    Leader: 2   Replicas: 0,1,2 Isr: 2,0,1

See also: the part of the official documentation that describes how
to increase the replication factor.

More Related Contents:

  1. How to send final kafka-streams aggregation result of a time windowed KTable?
  2. Streamparse wordcount example
  3. Difference between session.timeout.ms and max.poll.interval.ms for Kafka >= 0.10.1
  4. Is there a way to purge the topic in Kafka?
  5. How does an offset expire for an Apache Kafka consumer group?
  6. How to commit manually with Kafka Stream?
  7. Increase the number of messages read by a Kafka consumer in a single poll
  8. Kafka how to read from __consumer_offsets topic
  9. Delete message after consuming it in KAFKA
  10. Understanding Kafka Topics and Partitions
  11. When to use RabbitMQ over Kafka? [closed]
  12. Delete topic in Kafka 0.8.1.1
  13. Kafka Stream Suppress session-windowed-aggregation
  14. Use Confluent Hub without Confluent Platform installation
  15. How to change start offset for topic?
  16. Kafka 0.11 how to reset offsets
  17. How can I send large messages with Kafka (over 15MB)?
  18. Using Kafka as a (CQRS) Eventstore. Good idea?
  19. How to write spark streaming DF to Kafka topic
  20. Writing Custom Kafka Serializer
  21. Kafka server configuration – listeners vs. advertised.listeners
  22. How to manually set group.id and commit kafka offsets in spark structured streaming?
  23. What is a simple, effective way to debug custom Kafka connectors?
  24. Connect to Kafka on host from Docker (ksqlDB)
  25. SyntaxError on “self.async” when running python kafka producer
  26. Dynamically connecting a Kafka input stream to multiple output streams
  27. How to integrate Oracle and Kafka
  28. Can a Kafka consumer(0.8.2.2) read messages in batch
  29. How can I delete a topic in Apache Kafka? [duplicate]
  30. removing a kafka consumer group in zookeeper

Leave a Comment