Kafka: Running multiple instances of Kafka in the same host


In situations where you want run a cluster of brokers, but do not have spare hosts, you may run multiple brokers nodes in the same server host. For this you will require seperate copies of “config/server.properties” with unique values for the following variables in it:

  • broker.id
  • port
  • log.dir

Once you have these separate copies of “config/server.properties” files with distinct values of the above three, you may start up each broker instance using the same command:

  • bin/kafka-server-start.sh config/server1.properties
  • bin/kafka-server-start.sh config/server1.properties
  • etc