java kafka:提交偏移失败,可重试异常。您应该重试提交偏移量
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/49578138/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
kafka: Commit offsets failed with retriable exception. You should retry committing offsets
提问by enator
[o.a.k.c.c.i.ConsumerCoordinator] [Auto offset commit failed for group consumer-group: Commit offsets failed with retriable exception. You should retry committing offsets.] []
[oakcciConsumerCoordinator] [组消费者组的自动偏移提交失败:提交偏移失败,可重试异常。您应该重试提交偏移量。] []
Why does this error come in kafka consumer? what does this mean?
为什么这个错误会出现在 kafka 消费者中?这是什么意思?
The consumer properties I am using are:
我正在使用的消费者属性是:
fetch.min.bytes:1
enable.auto.commit:true
auto.offset.reset:latest
auto.commit.interval.ms:5000
request.timeout.ms:300000
session.timeout.ms:20000
max.poll.interval.ms:600000
max.poll.records:500
max.partition.fetch.bytes:10485760
What is the reason for that error to come? I am guessing the consumer is doing duplicated work right now (polling same message again) because of this error.
出现该错误的原因是什么?我猜因为这个错误,消费者现在正在做重复的工作(再次轮询相同的消息)。
I am neither using consumer.commitAsync()
or consumer.commitSync()
我既不使用consumer.commitAsync()
也不consumer.commitSync()
回答by Andremoniy
Consumer gives this error in case if it catches an instance of RetriableException
.
如果消费者捕获到 的实例,则会出现此错误RetriableException
。
The reasons for it might be various:
其原因可能是多种多样的:
- if coordinator is still loading the group metadata
- if the group metadata topic has not been created yet
- if network or disk corruption happens, or miscellaneous disk-related or network-related IOException occurred when handling a request
- if server disconnected before a request could be completed
- if the client's metadata is out of date
- if there is no currently available leader for the given partition
- if no brokers were available to complete a request
- 如果协调器仍在加载组元数据
- 如果尚未创建组元数据主题
- 如果发生网络或磁盘损坏,或者在处理请求时发生杂项磁盘相关或网络相关的 IOException
- 如果服务器在请求完成之前断开连接
- 如果客户端的元数据已过期
- 如果给定分区当前没有可用的领导者
- 如果没有经纪人可以完成请求
As you can see from the list above, all these errors couldbe temporary issues, that is why it is suggested to retry the request.
从上面的列表中可以看出,所有这些错误都可能是临时问题,这就是建议重试请求的原因。