Java 创建 Kafka 主题时出错:- 复制因子大于可用代理

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/28106080/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-11 05:36:55  来源:igfitidea点击:

Error creating Kafka topic :- replication factor larger than available brokers

javascalaapache-kafkakafka-consumer-api

提问by Count

I am trying to create a kafka topic via AdminCommand using below code Source

我正在尝试使用以下代码通过 AdminCommand 创建一个 kafka 主题

 ZkClient zkClient = new ZkClient(kafkaHost, 10000, 10000, ZKStringSerializer$.MODULE$);
    AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

But getting the below exception

但得到以下异常

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

However, I am able to create the topic using shell command .

但是,我可以使用 shell command 创建主题。

采纳答案by Jaya Ananthram

In your code,

在您的代码中,

 AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

The fourth argument is the replication factor. So you are trying to create a topic with a name of pa_reliancepoc_telecom_usageeventwith partition count of 10and replication of 2. So two kafka brokersshould be available while creating the topic. If less than two is available then you will get the following exception.

第四个参数是复制因子。因此,您正在尝试创建一个名称为pa_reliancepoc_telecom_usageeventwith partitioncount of 10replication of 2. 所以two kafka brokers在创建主题时应该可用。如果可用的少于两个,那么您将收到以下异常。

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

Make sure that you are running kafka cluster with two broker nodesand the two nodes should be alive while creating the topic.

确保您正在运行kafka cluster with two broker nodes并且两个节点should be alive while creating the topic.

To run kafka in cluster refer Step 6 in this link

要在集群中运行 kafka,请参阅此链接中的第 6 步

回答by user2720864

configure your local machine to have multiple brokers up and running in case you decide to keep your replication_factor > 1.

将本地计算机配置为启动并运行多个代理,以防您决定保留replication_factor > 1.

You can do that by simply having multiple copies of the server.propertiesfile. e.g
server-1.properties& server-2.properties

您可以通过简单地拥有server.properties文件的多个副本来做到这一点 。例如
server-1.properties&server-2.properties

You need to then specify different broker.id& portin each of those files to make them unique ..

然后,您需要在每个文件中指定不同的broker.id&port以使它们唯一..

   config/server-1.properties:
     broker.id=1
     port=9093
     log.dir=/tmp/kafka-logs-1

   config/server-2.properties:
     broker.id=2
     port=9094
     log.dir=/tmp/kafka-logs-2

And then start multiple instances with following commands

然后使用以下命令启动多个实例

> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &

For more details check Step 6: Setting up a multi-broker cluster

有关更多详细信息,请查看第 6 步:设置多代理集群

回答by Ananth Joshi

I faced the same issue while setting up multiple brokers.

我在设置多个代理时遇到了同样的问题。

The step I was missing was:

我缺少的步骤是:

While editing the config file: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1

编辑配置文件时: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1

need to update the LOG BASICS section as well(see below):

还需要更新 LOG BASICS 部分(见下文):

####################### 日志基础

A comma separated list of directories under which to store log files

以逗号分隔的目录列表,用于存储日志文件

log.dirs=/tmp/kafka-logs-1