pandas python del 没有释放所有内存
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/35121659/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
python del not freeing all the memory
提问by user2725109
In my python program, I use pandas
to read a csv file and store in memory:
在我的 python 程序中,我pandas
用来读取一个 csv 文件并存储在内存中:
data = pandas.read_csv('data.csv')
Before running the above command I check the free memory with free -m
and the output is 1704
. After running the above command the output is 729
. I run
在运行上述命令之前,我检查了可用内存,free -m
输出为1704
. 运行上述命令后,输出为729
. 我跑
del(data)
to free the memory used by data
. Now when I check the free memory the output is 1093
which is much less than the original 1704
. Where did the rest go? How can I free it? I'm running all these in ipython
and even exiting ipython doesn't free up that memory.
释放data
. 现在,当我检查可用内存时,输出1093
比原始1704
. 剩下的去哪儿了?我怎样才能释放它?我正在运行所有这些ipython
,甚至退出 ipython 也不会释放该内存。
Thanks.
谢谢。
回答by 22degrees
"Exiting ipython doesn't free up that memory" means that you're seeing the effect at the OS level. You're probably seeing the effect of memory caching. That data is saved in memory until the space is needed or the file is read again - instead of reading the file from disk it will be read from the 'inactive' section of memory.
“退出 ipython 不会释放该内存”意味着您在操作系统级别看到了效果。您可能会看到内存缓存的效果。该数据保存在内存中,直到需要空间或再次读取文件 - 而不是从磁盘读取文件,而是从内存的“非活动”部分读取。
Assuming you're on some unix/linux variant, for details about ram usage
假设您使用的是某种 unix/linux 变体,以了解有关 ram 使用情况的详细信息
cat /proc/meminfo
Your available memory is Free + Inactive. The Free is available now, and the OS will reclaim space from Inactive as necessary. I keep the following alias (in csh) around to check
您的可用内存为空闲 + 非活动。Free 现在可用,操作系统将根据需要从 Inactive 中回收空间。我保留以下别名(在 csh 中)以进行检查
alias 'freemem' cat /proc/meminfo | grep -i 'inactive:\|memfree:'
Python's del will remove your variable from the namespace. So if that memory isn't referenced anywhere else in your code, python's garbage collection will kick in and clear data from memory. Or if you want to force it:
Python 的 del 将从命名空间中删除您的变量。因此,如果该内存未在您的代码中的任何其他地方引用,python 的垃圾收集将启动并从内存中清除数据。或者,如果你想强制它:
import gc
foo = range(10000000)
del foo
gc.collect()
Regardless of all this, the OS will keep the file contents cached so it doesn't have to do redundant disk reads when you try to read that same file again.
无论如何,操作系统都会缓存文件内容,这样当您再次尝试读取同一个文件时,它就不必进行冗余磁盘读取。
回答by user3591723
del
is not C's free
or C++'s delete
del
不是 Cfree
或 C++delete
6.5. The del statement
del_stmt ::= "del" target_list
Deletion is recursively defined very similar to the way assignment is defined. Rather than spelling it out in full details, here are some hints.
Deletion of a target list recursively deletes each target, from left to right.
Deletion of a name removes the binding of that name from the local or global namespace, depending on whether the name occurs in a global statement in the same code block. If the name is unbound, a NameError exception will be raised.
It is illegal to delete a name from the local namespace if it occurs as a free variable in a nested block.
Deletion of attribute references, subscriptions and slicings is passed to the primary object involved; deletion of a slicing is in general equivalent to assignment of an empty slice of the right type (but even this is determined by the sliced object).
6.5. 德尔声明
del_stmt ::= "del" target_list
删除的递归定义与赋值的定义方式非常相似。这里没有详细说明,而是一些提示。
删除目标列表会从左到右递归地删除每个目标。
删除名称会从本地或全局名称空间中删除该名称的绑定,具体取决于该名称是否出现在同一代码块中的全局语句中。如果名称未绑定,则会引发 NameError 异常。
如果某个名称作为嵌套块中的自由变量出现,则从本地命名空间中删除该名称是非法的。
属性引用、订阅和切片的删除传递给所涉及的主要对象;切片的删除通常等同于分配正确类型的空切片(但即使这也是由切片对象决定的)。
You're not freeing the memory. You're telling python you're done with that variable.
你没有释放内存。你告诉python你已经完成了那个变量。