Java 从多个应用程序/进程记录到单个日志文件

Logging from multiple apps/processes to a single log file

提问by Andrew

Our app servers (weblogic) all use log4j to log to the same file on a network share. On top of this we have all web apps in a managed server logging errors to a common error.log. I can't imagine this is a good idea but wanted to hear from some pros. I know that each web app has its own classloader, so any thread synchronization only occurs within the app. So what happens when multiple processes start converging on a single log file? Can we expect interspersed log statements? Performance problems? What about multiple web apps logging to a common log file? The environment is Solaris.

我们的应用服务器 (weblogic) 都使用 log4j 登录到网络共享上的同一个文件。最重要的是，我们将托管服务器中的所有 Web 应用程序记录到常见的 error.log 中。我无法想象这是一个好主意，但想听听一些专业人士的意见。我知道每个 Web 应用程序都有自己的类加载器，因此任何线程同步都只发生在应用程序内。那么当多个进程开始汇聚在一个日志文件上时会发生什么？我们可以期待穿插的日志语句吗？性能问题？多个 Web 应用程序记录到一个公共日志文件怎么样？环境是Solaris。

采纳答案by pajton

This is generaly bad idea to have not synchronized write access to a file and certainly bad programming practice. The only case it might work is an append to a file on local machine - everybody just adds lines at the end of file.

没有同步对文件的写访问通常是个坏主意，当然也是糟糕的编程实践。它可能工作的唯一情况是附加到本地机器上的文件 - 每个人都只是在文件末尾添加行。

But, since your file is on the network share, it will probably quickly turn into garbage. You didn't tell which distributed filesystem you are using, but for NFS you can find following explanation on open(2) man page:

但是，由于您的文件位于网络共享上，它可能很快就会变成垃圾。您没有说明您使用的是哪个分布式文件系统，但对于 NFS，您可以在 open(2) 手册页上找到以下说明：

O_APPEND The file is opened in append mode. Before each write(), the file offset is positioned at the end of the file, as if with lseek(). O_APPEND may lead to corrupted files on NFS file systems if more than one process appends data to a file at once. This is because NFS does not support appending to a file, so the client kernel has to simulate it, which can't be done without a race condition.

O_APPEND 文件以追加模式打开。在每次 write() 之前，文件偏移量位于文件末尾，就像使用 lseek() 一样。如果多个进程同时将数据附加到一个文件，O_APPEND 可能会导致 NFS 文件系统上的文件损坏。这是因为 NFS 不支持附加到文件，所以客户端内核必须模拟它，这在没有竞争条件的情况下是无法完成的。

Of course this is C, but since Java is implemented in C it cannot do any better than that (at least not with regard to system calls:-)).

当然这是 C，但由于 Java 是用 C 实现的，所以它不能做得比这更好（至少在系统调用方面没有：-)）。

回答by nathan

We use org.apache.log4j.net.SyslogAppender to log to a single machine using syslog, and it has worked well for us. I would recommend looking into that as an alternative.

我们使用 org.apache.log4j.net.SyslogAppender 使用 syslog 登录到一台机器，它对我们来说效果很好。我建议将其作为替代方案进行研究。

回答by Yishai

This looks like a really bad idea (corrupt logs, uncertainty of the source of a given log entry are two reasons that come to mind). If you are using Log4j in Weblogic like this, I suggest doing it by-the-book. This will allow you to use one file for the whole app server without any issues.

这看起来是一个非常糟糕的主意（损坏的日志、给定日志条目来源的不确定性是我想到的两个原因）。如果您像这样在 Weblogic 中使用 Log4j，我建议您按书进行。这将允许您为整个应用服务器使用一个文件而不会出现任何问题。

The suggestion of syncronizing the log writing makes no sense to me, as you would be basically blocking all applications in the app server when they write a log. If logging is frequent, that will slow everything down significantly.

同步日志写入的建议对我来说毫无意义，因为当它们写入日志时，您基本上会阻止应用服务器中的所有应用程序。如果日志记录频繁，这将显着减慢一切。

As for multiple app servers, you need to use something other than file based logging if you want them all consolidated. There are a few ways to do this, one is to log to different files and have a different process combine them, but the better option is probably to use a network based logging repository, using Log4j's SocketAppender or some other method (nathan mentions SyslogAppender which is great if you want a Syslog) to ensure that the file access does not get corrupted.

对于多个应用程序服务器，如果您希望将它们全部合并，则需要使用基于文件的日志记录以外的其他内容。有几种方法可以做到这一点，一种是记录到不同的文件并让不同的进程组合它们，但更好的选择可能是使用基于网络的日志记录存储库，使用 Log4j 的 SocketAppender 或其他一些方法（nathan 提到 SyslogAppender，如果你想要一个 Syslog 是很好的）以确保文件访问不会被破坏。

回答by Will Iverson

Best case I would imagine you have a potential performance problem with sync access to the log file resource. Worst case would be some of the scenarios you mention.

最好的情况是，我认为您在同步访问日志文件资源时存在潜在的性能问题。最坏的情况是你提到的一些场景。

I would ask two Qs: a) what was the desired goal with this config and b) can you find a better technical solution to achieve the goal.

我会问两个问题：a）这个配置的预期目标是什么，b）你能找到更好的技术解决方案来实现目标。

My guess is that you have a sysadmin who wants to get a "single log file for the system." The file is thrown on the network share to make it easy to get to. The better answer for the goal might be multiple files, local to each system and something like http://www.splunk.com/for a nice monitor.

我的猜测是您的系统管理员想要获得“系统的单个日志文件”。该文件被扔到网络共享上，以便于访问。该目标的更好答案可能是多个文件，每个系统本地的文件，以及类似http://www.splunk.com/ 的文件，以获得一个不错的监视器。

回答by Thorbj?rn Ravn Andersen

If at all possible use a different file for each instance. This will give the best result with the least effort.

如果可能的话，为每个实例使用不同的文件。这将以最少的努力获得最好的结果。

The logback alternative to log4j has a prudent mode to its log writer which explicitly jump through hoops to ensure that new things are written at the end of file. I don't think that will work on network shares then.

log4j 的 logback 替代方案对其日志编写器具有谨慎模式，该模式显式跳过箍以确保在文件末尾写入新内容。我认为这不适用于网络共享。

If you MUST have a central logging location, then consider setting up a server accepting log events and writing them to the appropriate files. This will ensure it is only a single process actually accessing the file system, and will let the JVM help all it can in terms of synchronization etc.

如果你必须有一个中央日志位置，那么考虑设置一个服务器来接受日志事件并将它们写入适当的文件。这将确保它只是一个实际访问文件系统的进程，并让 JVM 在同步等方面尽其所能。

回答by Ceki

In prudent modelogback will safely handle multiple JVMs possibly on different hosts writing to the same network-shared file. It can even deal with temporary network failures. Performance should be quite acceptable for few nodes, say 4 or less. For 5 or more nodes all logging heavily you may notice a performance hit.

在谨慎模式下，logback 将安全地处理可能在不同主机上写入同一个网络共享文件的多个 JVM。它甚至可以处理临时网络故障。对于少数节点（例如 4 个或更少），性能应该是完全可以接受的。对于 5 个或更多节点的所有日志记录，您可能会注意到性能下降。

回答by Gautam Dey

We have a requirement where we need to produce a single file from all the managed server running the same application . we developed a java logging server which opens ups a port and listen for log event. we used the log4j socket appenderto write log event to the same port and created a single file.

我们需要从运行相同应用程序的所有托管服务器生成单个文件。我们开发了一个 java 日志服务器，它打开一个端口并监听日志事件。我们使用log4j 套接字附加程序将日志事件写入同一端口并创建了一个文件。