python中打开的文件太多
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/4386482/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Too many open files in python
提问by dmorlock
I wrote kind of a test suite which is heavily file intensive. After some time (2h) I get an IOError: [Errno 24] Too many open files: '/tmp/tmpxsqYPm'. I double checked all file handles whether I close them again. But the error still exists.
我写了一种文件密集型的测试套件。一段时间后（2 小时），我得到一个IOError: [Errno 24] Too many open files: '/tmp/tmpxsqYPm'. 我仔细检查了所有文件句柄是否再次关闭它们。但是错误仍然存在。
I tried to figure out the number of allowed file descriptors using resource.RLIMIT_NOFILEand the number of currently opened file desciptors:
我试图找出允许使用resource.RLIMIT_NOFILE的文件描述符的数量和当前打开的文件描述符的数量：
def get_open_fds():
fds = []
for fd in range(3,resource.RLIMIT_NOFILE):
try:
flags = fcntl.fcntl(fd, fcntl.F_GETFD)
except IOError:
continue
fds.append(fd)
return fds
So if I run the following test:
因此，如果我运行以下测试：
print get_open_fds()
for i in range(0,100):
f = open("/tmp/test_%i" % i, "w")
f.write("test")
print get_open_fds()
I get this output:
我得到这个输出：
[]
/tmp/test_0
[3]
/tmp/test_1
[4]
/tmp/test_2
[3]
/tmp/test_3
[4]
/tmp/test_4
[3]
/tmp/test_5
[4] ...
That's strange, I expected an increasing number of opened file descriptors. Is my script correct?
这很奇怪，我预计打开的文件描述符数量会增加。我的脚本正确吗？
I'm using python's logger and subprocess. Could that be the reason for my fd leak?
我正在使用 python 的记录器和子进程。这可能是我的 fd 泄漏的原因吗？
Thanks, Daniel
谢谢，丹尼尔
采纳答案by Ignacio Vazquez-Abrams
Your test script overwrites feach iteration, which means that the file will get closed each time. Both logging to files and subprocesswith pipes use up descriptors, which can lead to exhaustion.
您的测试脚本会覆盖f每次迭代，这意味着文件每次都会关闭。记录到文件和subprocess使用管道都会消耗描述符，这可能会导致耗尽。
回答by Peter Buckner
resource.RLIMIT_NOFILE is indeed 7, but that's an index into resource.getrlimit(), not the limit itself... resource.getrlimit(resource.RLIMIT_NOFILE) is what you want your top range() to be
resource.RLIMIT_NOFILE 确实是 7，但这是对 resource.getrlimit() 的索引，而不是限制本身...... resource.getrlimit(resource.RLIMIT_NOFILE) 是你想要的最高范围()
回答by blueFast
The corrected code is:
更正后的代码是：
import resource
import fcntl
import os
def get_open_fds():
fds = []
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
for fd in range(0, soft):
try:
flags = fcntl.fcntl(fd, fcntl.F_GETFD)
except IOError:
continue
fds.append(fd)
return fds
def get_file_names_from_file_number(fds):
names = []
for fd in fds:
names.append(os.readlink('/proc/self/fd/%d' % fd))
return names
fds = get_open_fds()
print get_file_names_from_file_number(fds)
