Java IOException“打开的文件太多”
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/4289447/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Java IOException "Too many open files"
提问by dpsthree
I doing some file I/O with multiple files (writing to 19 files, it so happens). After writing to them a few hundred times I get the Java IOException
: Too many open files
. But I actually have only a few files opened at once. What is the problem here? I can verify that the writes were successful.
我对多个文件进行了一些文件 I/O(写入 19 个文件,确实如此)。在给他们写了几百次之后,我得到了 Java IOException
: Too many open files
。但实际上我一次只打开了几个文件。这里有什么问题?我可以验证写入是否成功。
采纳答案by Stephen C
On Linux and other UNIX / UNIX-like platforms, the OS places a limit on the number of open file descriptors that a process may have at any given time. In the old days, this limit used to be hardwired1, and relatively small. These days it is much larger (hundreds / thousands), and subject to a "soft" per-process configurable resource limit. (Look up the ulimit
shell builtin ...)
在 Linux 和其他 UNIX / 类 UNIX 平台上,操作系统对进程在任何给定时间可能拥有的打开文件描述符的数量进行了限制。在过去,这个限制曾经是硬连线1,而且相对较小。现在它要大得多(成百上千),并且受到每个进程的“软”可配置资源限制。(查找ulimit
内置的shell ...)
Your Java application must be exceeding the per-process file descriptor limit.
您的 Java 应用程序必须超出每个进程的文件描述符限制。
You say that you have 19 files open, and that after a few hundred times you get an IOException saying "too many files open". Now this particular exception can ONLY happen when a new file descriptor is requested; i.e. when you are openinga file (or a pipe or a socket). You can verifythis by printing the stacktrace for the IOException.
你说你打开了 19 个文件,几百次后你得到一个 IOException 说“打开的文件太多”。现在这个特殊的异常只会在请求一个新的文件描述符时发生;即当您打开文件(或管道或套接字)时。您可以通过打印 IOException 的堆栈跟踪来验证这一点。
Unless your application is being run with a small resource limit (which seems unlikely), it follows that it must be repeatedly opening files / sockets / pipes, and failing to close them. Find out why that is happening and you should be able to figure out what to do about it.
除非您的应用程序以较小的资源限制运行(这似乎不太可能),否则它必须重复打开文件/套接字/管道,但无法关闭它们。找出发生这种情况的原因,您应该能够弄清楚该怎么做。
FYI, the following pattern is a safe way to write to files that is guaranteed not to leak file descriptors.
仅供参考,以下模式是一种写入文件的安全方式,可保证不会泄漏文件描述符。
Writer w = new FileWriter(...);
try {
// write stuff to the file
} finally {
try {
w.close();
} catch (IOException ex) {
// Log error writing file and bail out.
}
}
1 - Hardwired, as in compiled into the kernel. Changing the number of available fd slots required a recompilation ... and could result in less memory being available for other things. In the days when Unix commonly ran on 16-bit machines, these things really mattered.
1 - 硬连线,如编译到内核中。更改可用 fd 插槽的数量需要重新编译......并且可能导致更少的内存可用于其他事情。在 Unix 通常在 16 位机器上运行的日子里,这些事情真的很重要。
UPDATE
更新
The Java 7 way is more concise:
Java 7 的方式更简洁:
try (Writer w = new FileWriter(...)) {
// write stuff to the file
} // the `w` resource is automatically closed
UPDATE 2
更新 2
Apparently you can also encounter a "too many files open" while attempting to run an external program. The basic cause is as described above. However, the reason that you encounter this in exec(...)
is that the JVM is attempting to create "pipe" file descriptors that will be connected to the external application's standard input / output / error.
显然,您在尝试运行外部程序时也会遇到“打开的文件太多”。基本原因如上所述。但是,您遇到这种情况的原因exec(...)
是 JVM 正在尝试创建“管道”文件描述符,这些描述符将连接到外部应用程序的标准输入/输出/错误。
回答by Falmarri
You're obviously not closing your file descriptors before opening new ones. Are you on windows or linux?
在打开新文件描述符之前,您显然没有关闭文件描述符。你是windows还是linux?
回答by Xiè Jìléi
Recently, I had a program batch processing files, I have certainly closed each file in the loop, but the error still there.
最近有一个程序批处理文件,我当然已经关闭了循环中的每个文件,但错误仍然存在。
And later, I resolved this problem by garbage collect eagerly every hundreds of files:
后来,我通过每隔数百个文件进行垃圾收集来解决这个问题:
int index;
while () {
try {
// do with outputStream...
} finally {
out.close();
}
if (index++ % 100 = 0)
System.gc();
}
回答by pk10
For UNIX:
对于 UNIX:
As Stephen C has suggested, changing the maximum file descriptor value to a higher value avoids this problem.
正如 Stephen C 所建议的,将最大文件描述符值更改为更高的值可以避免此问题。
Try looking at your present file descriptor capacity:
尝试查看您当前的文件描述符容量:
$ ulimit -n
Then change the limit according to your requirements.
然后根据您的要求更改限制。
$ ulimit -n <value>
Note that this just changes the limits in the current shell and any child / descendant process. To make the change "stick" you need to put it into the relevant shell script or initialization file.
请注意,这只会更改当前 shell 和任何子/后代进程中的限制。要使更改“坚持”,您需要将其放入相关的 shell 脚本或初始化文件中。
回答by pk10
Although in most general cases the error is quite clearly that file handles have not been closed, I just encountered an instance with JDK7 on Linux that well... is sufficiently ****ed up to explain here.
尽管在大多数一般情况下,错误很明显是文件句柄尚未关闭,但我刚刚在 Linux 上遇到了一个带有 JDK7 的实例……这足以解释这里。
The program opened a FileOutputStream (fos), a BufferedOutputStream (bos) and a DataOutputStream (dos). After writing to the dataoutputstream, the dos was closed and I thought everything went fine.
该程序打开了一个 FileOutputStream (fos)、一个 BufferedOutputStream (bos) 和一个 DataOutputStream (dos)。写入数据输出流后,dos 关闭,我认为一切正常。
Internally however, the dos, tried to flush the bos, which returned a Disk Full error. That exception was eaten by the DataOutputStream, and as a consequence the underlying bos was not closed, hence the fos was still open.
然而,在内部,dos 试图刷新 bos,它返回磁盘已满错误。该异常被 DataOutputStream 吃掉了,因此底层 bos 没有关闭,因此 fos 仍然打开。
At a later stage that file was then renamed from (something with a .tmp) to its real name. Thereby, the java file descriptor trackers lost track of the original .tmp, yet it was still open !
在稍后阶段,该文件从(带有 .tmp 的东西)重命名为其真实名称。因此,java 文件描述符跟踪器丢失了原始 .tmp 的跟踪,但它仍然是打开的!
To solve this, I had to first flush the DataOutputStream myself, retrieve the IOException and close the FileOutputStream myself.
为了解决这个问题,我必须首先自己刷新 DataOutputStream,检索 IOException 并自己关闭 FileOutputStream。
I hope this helps someone.
我希望这可以帮助别人。