Kafka Broker | Command-line Options and Procedure

In this Apache Kafka tutorial, we are going to learn Kafka Broker. Kafka Broker manages the storage of messages in the topic(s). If Apache Kafka has more than one broker, that is what we call a Kafka cluster. Moreover, in this Kafka Broker Tutorial, we will learn how to start Kafka Broker and Kafka command-line Option.
So, let’s start Apache Kafka Broker.

Kafka Broker

Kafka Broker | Command-line Options and Procedure

Stay updated with latest technology trends
Join DataFlair on Telegram!!

What is Kafka Broker?

A Kafka broker is also known as Kafka server and a Kafka node. These all names are its synonyms. In simple words, a broker is a mediator between two. However, Kafka broker is more precisely described as a Message Broker which is responsible for mediating the conversation between different computer systems, guaranteeing delivery of the message to the correct parties.

Read Apache Kafka Security | Need and Components of Kafka

Hence, the Kafka cluster typically consists of multiple brokers. Kafka Cluster uses Zookeeper for maintaining the cluster state. A single Broker can handle thousands of reads and writes per second. Whereas, if there is no performance impact, each broker can handle TB of messages. In addition, to be very sure that ZooKeeper performs broker leader election.

Kafka Broker

Basically, a broker in Kafka is modeled as KafkaServer, which hosts topics. Here, given topics are always partitioned across brokers, in a cluster a single broker hosts topic partitions of one or more topics actually, even when a topic is only partitioned to just a single partition.

In addition, producers send a message to a broker, after receiving broker stores them on disk keyed by unique offset. Moreover, by topic, partition and offset a broker allows consumers to fetch messages. However, brokers can create a cluster just by sharing information with each other directly or indirectly using Zookeeper. Also, we can say Kafka cluster has exactly one broker which acts as the Controller. By using kafka-server-start.sh script, we can start a single Kafka broker.

How to Start Kafka Broker?

  • Start Zookeeper.
./bin/zookeeper-server-start.sh config/zookeeper.properties

We can start a Kafka server, only when Zookeeper is up and running(that will connect to Zookeeper).

./bin/kafka-server-start.sh config/server.properties
kafka-server-start.sh script

Hence, kafka-server-start.sh starts a broker.

$ ./bin/kafka-server-start.sh
USAGE: ./bin/kafka-server-start.sh [-daemon] server.properties [--override property=value]*

Note- make sure that Zookeeper is up and running before you run kafka-server-start.sh. Moreover, use zookeeper-server-start shell script.

Have a look at Apache Kafka + Spark Streaming Integration

kafka-server-start.sh uses config/log4j.properties for logging configuration that we can override using KAFKA_LOG4J_OPTS environment variable.


kafka-server-start.sh accepts KAFKA_HEAP_OPTS and EXTRA_ARGS environment variables.

Kafka Command-line Options


Defaults to kafkaServer when in daemon mode.


Enabled when in daemon mode.


Enables daemon mode.

–override property=value

A value that should override the value set for property in server.properties file.

Let’s revise Apache Kafka Streams | Stream Processing Topology

$ ./bin/kafka-server-start.sh config/server.properties --override broker.id=100
INFO [KafkaServer id=100] started (kafka.server.KafkaServer)


Hence, we have learned the complete Kafka Broker, along with commands to start it. However, if any doubt occurs, feel free to ask in the comment section.

See also – 

Kafka- serialization and deserialization

For reference

6 Responses

  1. Andrei says:

    On the right side of broker shouldn’t be consumers on the picture?

  2. Kishan says:

    I am a bit confuse about Kafka broker , basically I understood Broker is something kind of server which hosts the Kafka Topic and Broker can be multiple that is why kafka cluster exists, and Zookeeper manage all the Brokers if my understanding is correct than,
    please tell me How I can create multiple Broker that host different topics.

  3. Kishan says:

    Also I want some clarification that what do you mean by , brokers can create a cluster just by sharing information with each other directly or indirectly using Zookeeper.
    Is it mean that Broker can create another Broker so that they can form Cluster inside the Kafka, I am a bit confused.

  4. Vivek Wadhawan says:

    Is Controller and leader is same in Kafka Broker?

  5. anantak says:

    How to add Kafka broker by using command line any command for the same?

Leave a Reply

