java Kafka Consumer 向控制台输出过多的 DEBUG 语句(ecilpse)

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/44129303/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-11-03 07:57:59  来源:igfitidea点击:

Kafka Consumer outputs excessive DEBUG statements to console (ecilpse)

javaloggingapache-kafkalog4jkafka-consumer-api

提问by Bay Wei Heng

I'm running some sample code from http://www.javaworld.com/article/3060078/big-data/big-data-messaging-with-kafka-part-1.html?page=2, and the kafkaconsumer is consuming from topic as desired, but every poll results in print (to std out) of many debug logs, which I don't want.

我正在从http://www.javaworld.com/article/3060078/big-data/big-data-messaging-with-kafka-part-1.html?page=2运行一些示例代码,而 kafkaconsumer 是根据需要从主题中消费,但每次轮询都会导致打印(到标准输出)许多调试日志,这是我不想要的。

I have tried changing all INFO and DEBUG to ERROR (even did a grep to make sure) in /config/log4j.properties, in particular setting log4j.logger.kafka=ERROR, kafkaAppender, but the problem persists. I referred to How to configure logging for Kafka producers?, and adopted the solution there, but perhaps the situation is different for consumers?

我已经尝试将所有 INFO 和 DEBUG 更改为 ERROR(甚至做了一个 grep 以确保)/config/log4j.properties,特别是设置log4j.logger.kafka=ERROR,kafkaAppender,但问题仍然存在。我提到了如何为 Kafka 生产者配置日志记录?,并采用了那里的解决方案,但也许消费者的情况不同?

The DEBUG messages all have a similar format:

DEBUG 消息都具有类似的格式:

[Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - Sending fetch for partitions... to broker... (id: 0 rack: null)

and are appearing at rate of 10 every second or so (changing poll argument to 1000 or even 10000 doesn't help, I tried)

并且以每秒 10 次左右的速度出现(将投票参数更改为 1000 甚至 10000 无济于事,我试过了)

Would really appreciate any help from any expert. Thanks in advance!

非常感谢任何专家的任何帮助。提前致谢!

Edit: Not sure if it matters, but I added BasicConfigurator.configure();to my main method, to resolve some other error occurring previously that stopped the Consumer from even starting.

编辑:不确定它是否重要,但我添加BasicConfigurator.configure();到我的主要方法中,以解决之前发生的一些其他错误,这些错误甚至阻止了消费者的启动。

采纳答案by diginoise

Just modify the logging level of the chatty class (chatty interaction). Since in your logs you see log entries originating from org.apache.kafka.clients.consumer.internals.Fetcheryou can simply adjust the logging level for that logger by adding following line to log4j.properties:

只需修改chatty 类的日志记录级别(chatty 交互)。由于在您的日志中您会看到源自您的日志条目,org.apache.kafka.clients.consumer.internals.Fetcher您可以通过将以 下行添加到以下行来简单地调整该记录器的日志记录级别log4j.properties

log4j.logger.org.apache.kafka.clients.consumer.internals.Fetcher=WARN

... or any wider catching logger since these are name spaced:

...或任何更广泛的捕获记录器,因为它们是名称间隔的:

# adjusting logging for entire Kafka
log4j.logger.org.apache.kafka=WARN

Hope this helps

希望这可以帮助

回答by Zouinkhi

create new config xml file

创建新的配置xml文件

src/main/resources/logback.xml

src/main/resources/logback.xml

<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <logger name="org.apache.kafka" level="WARN"/>
    <logger name="org.apache.kafka.common.metrics" level="WARN"/>
    <root level="warn">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

回答by user11488099

Not sure if you are talking about kafka-console-consumer commands, if yes this is what I did:

不确定您是否在谈论 kafka-console-consumer 命令,如果是,这就是我所做的:

[training@confluent-training-vm ~]$ cd /etc/kafka
[training@confluent-training-vm kafka]$ grep DEBUG *.properties
log4j.properties:# Change to DEBUG or TRACE to enable request logging
log4j.properties:# Access denials are logged at INFO level, change to DEBUG to also 
log allowed accesses
tools-log4j.properties:log4j.rootLogger=DEBUG, stderr

So, you just need to edit /etc/kafka/tools-log4j.propertiesfile and remove DEBUG(or replace it for exmaple by INFOand WARMlog levels on above line

因此,您只需要编辑/etc/kafka/tools-log4j.properties文件并删除DEBUG(或将其替换为上面一行的示例INFOWARM日志级别

tools-log4j.properties:log4j.rootLogger=INFO, WARM, stderr

回答by szedjani

I found the solution under another question which deals with Kafka producers, but it's essentially the same problem: https://stackoverflow.com/a/49532152/2380553

我在另一个处理 Kafka 生产者的问题下找到了解决方案,但本质上是相同的问题:https: //stackoverflow.com/a/49532152/2380553

So I just run the following 3 lines at the beginning of my program:

所以我只是在我的程序开始时运行以下 3 行:

org.apache.log4j.Logger.getLogger("org").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("akka").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("kafka").setLevel(Level.WARN);