Java Kafka Connect 耗尽堆空间
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/41964676/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Kafka Connect running out of heap space
提问by Robin Daugherty
After starting Kafka Connect (connect-standalone), my task fails immediately after starting with:
启动 Kafka Connect ( connect-standalone) 后，我的任务在启动后立即失败：
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:343)
at org.apache.kafka.common.network.Selector.poll(Selector.java:291)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:180)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:193)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:248)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1013)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:979)
at org.apache.kafka.connect.runtime.WorkerSinkTask.pollConsumer(WorkerSinkTask.java:316)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:222)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:170)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:142)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
There's a mention of heap space in some Kafka documentation, telling you to try it with "the default" and only modifying it if there are problems, but there are no instructions to modify the heap space.
一些 Kafka 文档中提到了堆空间，告诉您使用“默认值”尝试并仅在出现问题时修改它，但没有修改堆空间的说明。
采纳答案by Robin Daugherty
You can control the max and initial heap size by setting the KAFKA_HEAP_OPTSenvironment variable.
您可以通过设置KAFKA_HEAP_OPTS环境变量来控制最大和初始堆大小。
The following example sets a starting size of 512 MB and a maximum size of 1 GB:
以下示例设置起始大小为 512 MB，最大大小为 1 GB：
KAFKA_HEAP_OPTS="-Xms512m -Xmx1g" connect-standalone connect-worker.properties connect-s3-sink.properties
When running a Kafka command such as connect-standalone, the kafka-run-classscript is invoked, which sets a default heap size of 256 MBin the KAFKA_HEAP_OPTSenvironment variable if it is not already set.
运行 Kafka 命令（例如 ）时connect-standalone，kafka-run-class会调用该脚本，如果尚未设置，则会在环境变量中设置默认堆大小为 256 MBKAFKA_HEAP_OPTS。
回答by Sagarmatha
Even I was facing the issue could not start my producer and consumer for a given topic. Also deleted all unnecessary log files and topics.Even though that's not related to the issue.
即使我面临这个问题，也无法为给定的主题启动我的生产者和消费者。还删除了所有不必要的日志文件和主题。即使这与问题无关。
Changing the kafka-run-class.shdid not work for me. I changed the below files
改变kafka-run-class.sh对我不起作用。我更改了以下文件
kafka-console-consumer.sh
kafka-console-producer.sh
kafka-console-consumer.sh
kafka-console-producer.sh
and stopped getting OOM error. Both consumer and producer worked fine after this.
并停止收到 OOM 错误。在此之后，消费者和生产者都运行良好。
Increased the size to KAFKA_HEAP_OPTS="-Xmx1G"was 512m earlier.
将大小增加到KAFKA_HEAP_OPTS="-Xmx1G"之前的 512m。
回答by peedee
When you have Kafka problems with
当您遇到 Kafka 问题时
java.lang.OutOfMemoryError: Java heap space
it doesn't necessarily mean that it's a memory problem. Several Kafka admin tools like kafka-topics.shwill mask the true error with this when trying to connect to an SSL PORT. The true (masked) error is SSL handshake failed!
这并不一定意味着这是一个内存问题。kafka-topics.sh尝试连接到 SSL 端口时，一些 Kafka 管理工具（例如）将掩盖真正的错误。真正的（被掩盖的）错误是SSL handshake failed！
See this issue: https://issues.apache.org/jira/browse/KAFKA-4090
看到这个问题：https: //issues.apache.org/jira/browse/KAFKA-4090
The solution is to include a properties file in your command (for kafka-topics.shthis would be --command-config) and to absolutely include this line:
解决方案是在您的命令中包含一个属性文件（因为kafka-topics.sh这将是--command-config）并绝对包含以下行：
security.protocol=SSL
