Java 太多打开的文件:打开了多少,它们是什么,JVM可以打开多少

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/2272908/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-13 05:31:52  来源:igfitidea点击:

Too many open files: how many are open, what they are, and how many can the JVM open

javajvm

提问by Dima

I'm getting this exception in Java:

我在 Java 中遇到此异常:

java.io.FileNotFoundException: (Too many open files) 

I'm looking for the ways to eliminate this problem.

我正在寻找消除这个问题的方法。

This error obviously indicates that JVM has allocated too many handles and underlying OS won't let it have more. Either I've got leak somewhere with improperly closed connections/streams.

此错误显然表明 JVM 分配了太多句柄,底层操作系统不会让它拥有更多句柄。要么我在某处泄漏了不正确关闭的连接/流。

This process runs for days non-stop and eventually throws the exception. It repeatedly happens after 12-14 days of up-time.

此过程连续运行数天并最终抛出异常。它在正常运行 12-14 天后反复发生。

How do you fight this? Is there a way to get a list of allocated handles in JVM or track when it hits certain amount? I'd love to have them printed and see how it grows and when. I can't use a profiler because it's a production system and have difficulties to reproduce it in development. Any suggestion?

你怎么打这个?有没有办法在 JVM 中获取分配的句柄列表或在达到一定数量时进行跟踪?我很想把它们打印出来,看看它是如何生长的以及何时生长。我不能使用分析器,因为它是一个生产系统,并且在开发中很难复制它。有什么建议吗?

I am monitoring free heap size and raising an "alarm" when it approaches 1% of the total specified in -Xmx. I also know that if my thread count hits above 500, then something definitely goes out of hand. Now, is there a way to know thatmy JVM allocates too many handlesfrom OS and doesn't give them back, e.g. sockets, opened files, etc. If I'd knew that, I'd know where to look and when.

我正在监视空闲堆大小并在它接近 -Xmx 中指定的总数的 1% 时发出“警报”。我也知道,如果我的线程数超过 500,那么肯定会失控。现在,有没有办法知道我的 JVM从操作系统分配了太多句柄并且没有将它们返回,例如套接字、打开的文件等。如果我知道这一点,我就会知道在哪里查看以及何时查看。

回答by bramp

You didn't say which OS you are running on, but if you are running on Linux you can use the lsof command

你没有说你在哪个操作系统上运行,但如果你在 Linux 上运行,你可以使用 lsof 命令

lsof -p <pid of jvm>

That will list all the files opened by the JVM. Or if you are running on Windows you can Process Explorerwhich will show all the open files for all the processes.

这将列出 JVM 打开的所有文件。或者,如果您在 Windows 上运行,则可以使用Process Explorer,它将显示所有进程的所有打开文件。

Doing this will hopefully allow you to narrow down which bit of the code is keeping the files open.

这样做有望让您缩小代码的哪一部分保持文件打开的范围。

回答by phisch

Since you are on Linux, I'd suggest, that you check the /proc-Filesystem. Inside proc, you will find a folder with the PID of your process containing a folder calld 'fd'. If your process id is 1234, the path is be

由于您使用的是 Linux,我建议您检查 /proc-Filesystem。在 proc 中,您将找到一个带有进程 PID 的文件夹,其中包含一个名为“fd”的文件夹。如果您的进程 ID 为 1234,则路径为

/proc/1234/fd

Inside that folder, you will find links to all opened files (do a 'ls -l'). Usually, you can tell by the filename which library / code might open and not close the file.

在该文件夹中,您将找到指向所有打开文件的链接(执行“ls -l”)。通常,您可以通过文件名判断哪个库/代码可能会打开而不是关闭文件。

回答by gaboroncancio

You can change the limit of opened files by adding the following to /etc/security/limits.conf:

您可以通过添加以下内容来更改打开文件的限制/etc/security/limits.conf

* soft nofile 2048 # Set the limit according to your needs
* hard nofile 2048

Then you can reload the configuration using sysctl -pon the shell. Check this article.

然后您可以sysctl -p在 shell 上使用重新加载配置。检查这篇文章

Just for completeness you can verify what is the current limit for opened files using: ulimit -n

为了完整起见,您可以使用以下方法验证打开文件的当前限制: ulimit -n

回答by Jimilian

So, full answer (I combined answers from @phisch and @bramp). If you want to check all processes, you should use sudo. Also it's nice to save result to file - lsof is not cheap + this file could be useful for further investigation.

所以,完整的答案(我结合了@phisch 和@bramp 的答案)。如果要检查所有进程,则应使用sudo. 将结果保存到文件也很好 - lsof 并不便宜 + 此文件可能对进一步调查有用。

sudo lsof > lsof.log

Show bad guys (with UPDATEfrom @Arun's comment):

显示坏人(与UPDATE来自@阿伦的评论):

cat lsof.log | awk '{print  " "  " " }' | sort | uniq |awk '{ print  " " ; }' | sort -rn | uniq -c | sort -rn | head -5

    2687 114970 java
    131 127992 nginx
    109 128005 nginx
    105 127994 nginx
    103 128019 nginx

Save list of file descriptors to file as well:

将文件描述符列表也保存到文件:

sudo ls -l /proc/114970/fd > fd.log

Show top open files:

显示最常打开的文件:

cat fd | awk '{ print  }' | sort -rn | uniq -c | sort -rn | head -n20

回答by Sahan Jayasumana

If you are on MacOS

如果您使用的是 MacOS

sudo launchctl limit maxfiles <hard> <soft>
sudo launchctl limit maxfiles 1024 200000