如何为 Kafka 设置 Java 选项?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/27681511/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-11 04:50:23  来源:igfitidea点击:

How do I set the Java options for Kafka?

javaapache-kafka

提问by Luis Medina

I've been experimenting with Kafka and saw from the documentation on the main site that you're able to set different options for the jvm like heap size and the garbage collector that it uses:

我一直在尝试使用 Kafka,并从主站点上的文档中看到,您可以为 jvm 设置不同的选项,例如堆大小和它使用的垃圾收集器:

http://kafka.apache.org/documentation.html#java

http://kafka.apache.org/documentation.html#java

What it doesn't say, however, is how/where to set these options. The application comes with a /config directory containing a lot of files used for configuration purposes but none that are for Java. It also comes with a /bin directory containing a bunch of scripts for Kafka but again, nothing really indicating how to configure Java.

然而,它没有说的是如何/在哪里设置这些选项。该应用程序带有一个 /config 目录,其中包含许多用于配置目的的文件,但没有一个是用于 Java 的。它还带有一个 /bin 目录,其中包含一堆 Kafka 脚本,但同样,没有真正说明如何配置 Java。

So my question is, how do I configure the Java options that Kafka uses? Is it done through a file or is there a different way?

所以我的问题是,如何配置 Kafka 使用的 Java 选项?它是通过文件完成的还是有不同的方式?

采纳答案by benny.la

I disagree with the accepted answer. Modifying a script in the bindirectory is highly unrecommended. When upgrading Kafka to the next version, extracting the new binaries would override the changes made in the script.

我不同意接受的答案。bin极不推荐修改目录中的脚本。将 Kafka 升级到下一个版本时,提取新的二进制文件将覆盖在脚本中所做的更改。

The preferred way should be to set the environment variable KAFKA_HEAP_OPTSoutside the script.

首选方法应该是KAFKA_HEAP_OPTS在脚本之外设置环境变量。

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

If the var is set before starting Kafka via the script it will use the var instead of the default values defined in /bin/kafka-server-start.sh

如果在通过脚本启动 Kafka 之前设置了 var,它将使用 var 而不是定义的默认值 /bin/kafka-server-start.sh

回答by sol4me

You can pass java parameters from command line . E.g.

您可以从命令行传递 java 参数。例如

java -server -Xms3072m -Xmx3072m -XX:NewSize=256m -XX:MaxNewSize=256m  -classpath <long list of jars> foo.class

For a `production server config you can create a properties file or set them in Code by creating

对于`生产服务器配置,您可以创建一个属性文件或通过创建在代码中设置它们

 Properties props = new Properties();
 props.put("serializer.class", "kafka.serializer.StringEncoder");

And then supply them to producerConfig

然后将它们提供给 producerConfig

ProducerConfig config = new ProducerConfig(props);

回答by Salvador Dali

Another way to do this is by modifying information written in /bin/kafka-server-start.sh:

另一种方法是修改写入的信息/bin/kafka-server-start.sh

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

or in /bin/kafka-run-class.sh:

或在/bin/kafka-run-class.sh

KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseCompressedOops -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSScavengeBeforeRemark -XX:+DisableExplicitGC -Djava.awt.headless=true"

回答by mfedko

Looking at kafka-run-classh.sh - kafka uses the following variables:

查看 kafka-run-classh.sh - kafka 使用以下变量:

  • $KAFKA_HEAP_OPTS
  • $KAFKA_JVM_PERFORMANCE_OPTS
  • $KAFKA_GC_LOG_OPTS
  • $KAFKA_JMX_OPTS
  • $KAFKA_LOG4J_OPTS
  • $KAFKA_HEAP_OPTS
  • $KAFKA_JVM_PERFORMANCE_OPTS
  • $KAFKA_GC_LOG_OPTS
  • $KAFKA_JMX_OPTS
  • $KAFKA_LOG4J_OPTS

You can run it via:

您可以通过以下方式运行它:

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" 
export KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12346 -Dcom.sun.management.jmxremote.rmi.port=12346 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false" 
bin/kafka-server-start.sh -daemon config/server.properties

回答by H.?.T

You can also set this parameters via your service definition as shown below.

您还可以通过您的服务定义设置此参数,如下所示。

[Unit]
Requires=zookeeper.service
After=zookeeper.service

[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/home/kafka/kafka/bin/kafka-server-start.sh /home/kafka/kafka/config/server.properties > /home/kafka/kafka.log 2>&1'
ExecStop=/home/kafka/kafka/bin/kafka-server-stop.sh
Restart=on-abnormal
Environment="KAFKA_HEAP_OPTS=-Xmx4G"
Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.net.preferIPv4Stack=true"
Environment="JMX_PORT=9999"

[Install]
WantedBy=multi-user.target

回答by Alex Rewa

To add generic JVM settings (for example user timezone) you could use KAFKA_OPTSenvironment variable from kafka-run-class.sh:

要添加通用 JVM 设置(例如用户时区),您可以使用以下KAFKA_OPTS环境变量kafka-run-class.sh

# Generic jvm settings you want to add
if [ -z "$KAFKA_OPTS" ]; then
  KAFKA_OPTS=""
fi