Java 创建 Kafka 主题时出错:- 复制因子大于可用代理
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/28106080/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Error creating Kafka topic :- replication factor larger than available brokers
提问by Count
I am trying to create a kafka topic via AdminCommand using below code Source
我正在尝试使用以下代码源通过 AdminCommand 创建一个 kafka 主题
ZkClient zkClient = new ZkClient(kafkaHost, 10000, 10000, ZKStringSerializer$.MODULE$);
AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());
But getting the below exception
但得到以下异常
Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)
However, I am able to create the topic using shell command .
但是,我可以使用 shell command 创建主题。
采纳答案by Jaya Ananthram
In your code,
在您的代码中,
AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());
The fourth argument is the replication factor. So you are trying to create a topic with a name of pa_reliancepoc_telecom_usageevent
with partition count of 10
and replication of 2
. So two kafka brokers
should be available while creating the topic. If less than two is available then you will get the following exception.
第四个参数是复制因子。因此,您正在尝试创建一个名称为pa_reliancepoc_telecom_usageevent
with partitioncount of 10
和replication of 2
. 所以two kafka brokers
在创建主题时应该可用。如果可用的少于两个,那么您将收到以下异常。
Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)
Make sure that you are running kafka cluster with two broker nodes
and the two nodes should be alive while creating the topic.
确保您正在运行kafka cluster with two broker nodes
并且两个节点should be alive while creating the topic.
To run kafka in cluster refer Step 6 in this link
要在集群中运行 kafka,请参阅此链接中的第 6 步
回答by user2720864
configure your local machine to have multiple brokers up and running in case you decide to keep your replication_factor > 1
.
将本地计算机配置为启动并运行多个代理,以防您决定保留replication_factor > 1
.
You can do that by simply having multiple copies of the
server.properties
file. e.g server-1.properties
& server-2.properties
您可以通过简单地拥有server.properties
文件的多个副本来做到这一点
。例如server-1.properties
&server-2.properties
You need to then specify different broker.id
& port
in each of those files to make them unique ..
然后,您需要在每个文件中指定不同的broker.id
&port
以使它们唯一..
config/server-1.properties:
broker.id=1
port=9093
log.dir=/tmp/kafka-logs-1
config/server-2.properties:
broker.id=2
port=9094
log.dir=/tmp/kafka-logs-2
And then start multiple instances with following commands
然后使用以下命令启动多个实例
> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &
For more details check Step 6: Setting up a multi-broker cluster
有关更多详细信息,请查看第 6 步:设置多代理集群
回答by Ananth Joshi
I faced the same issue while setting up multiple brokers.
我在设置多个代理时遇到了同样的问题。
The step I was missing was:
我缺少的步骤是:
While editing the config file: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1
编辑配置文件时: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1
need to update the LOG BASICS section as well(see below):
还需要更新 LOG BASICS 部分(见下文):
####################### 日志基础A comma separated list of directories under which to store log files
以逗号分隔的目录列表,用于存储日志文件
log.dirs=/tmp/kafka-logs-1