java kafka:提交偏移失败,可重试异常。您应该重试提交偏移量

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/49578138/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-11-03 10:24:21  来源:igfitidea点击:

kafka: Commit offsets failed with retriable exception. You should retry committing offsets

javaapache-kafka

提问by enator

[o.a.k.c.c.i.ConsumerCoordinator] [Auto offset commit failed for group consumer-group: Commit offsets failed with retriable exception. You should retry committing offsets.] []

[oakcciConsumerCoordinator] [组消费者组的自动偏移提交失败:提交偏移失败,可重试异常。您应该重试提交偏移量。] []

Why does this error come in kafka consumer? what does this mean?

为什么这个错误会出现在 kafka 消费者中?这是什么意思?

The consumer properties I am using are:

我正在使用的消费者属性是:

fetch.min.bytes:1
enable.auto.commit:true
auto.offset.reset:latest
auto.commit.interval.ms:5000
request.timeout.ms:300000
session.timeout.ms:20000
max.poll.interval.ms:600000
max.poll.records:500
max.partition.fetch.bytes:10485760

What is the reason for that error to come? I am guessing the consumer is doing duplicated work right now (polling same message again) because of this error.

出现该错误的原因是什么?我猜因为这个错误,消费者现在正在做重复的工作(再次轮询相同的消息)。

I am neither using consumer.commitAsync()or consumer.commitSync()

我既不使用consumer.commitAsync()也不consumer.commitSync()

回答by Andremoniy

Consumer gives this error in case if it catches an instance of RetriableException.

如果消费者捕获到 的实例,则会出现此错误RetriableException

The reasons for it might be various:

其原因可能是多种多样的:

  • if coordinator is still loading the group metadata
  • if the group metadata topic has not been created yet
  • if network or disk corruption happens, or miscellaneous disk-related or network-related IOException occurred when handling a request
  • if server disconnected before a request could be completed
  • if the client's metadata is out of date
  • if there is no currently available leader for the given partition
  • if no brokers were available to complete a request
  • 如果协调器仍在加载组元数据
  • 如果尚未创建组元数据主题
  • 如果发生网络或磁盘损坏,或者在处理请求时发生杂项磁盘相关或网络相关的 IOException
  • 如果服务器在请求完成之前断开连接
  • 如果客户端的元数据已过期
  • 如果给定分区当前没有可用的领导者
  • 如果没有经纪人可以完成请求

As you can see from the list above, all these errors couldbe temporary issues, that is why it is suggested to retry the request.

从上面的列表中可以看出,所有这些错误都可能是临时问题,这就是建议重试请求的原因。