Python 是否将 numpy 数组设置为无空闲内存?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/35316728/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Does setting numpy arrays to None free memory?
提问by daniel451
I have hundreds of really larges matrices, like (600, 800) or (3, 600, 800) shape'd ones.
我有数百个非常大的矩阵,比如 (600, 800) 或 (3, 600, 800) 形状的矩阵。
Therefore I want to de-allocate the memory used as soon as I don't really need something anymore.
因此,一旦我真的不再需要某些东西,我就想取消分配使用的内存。
I thought:
我想:
some_matrix = None
Should do the job, or is just the reference set to None but somewhere in the Memory the space still allocated? (like preserving the allocated space for some re-initialization of some_matrix
in the future)
应该完成这项工作,还是只是将引用设置为 None 但在内存中的某个地方仍然分配了空间?(例如保留分配的空间以some_matrix
备将来重新初始化)
Additionally: sometimes I am slicing through the matrices, calculated something and put the values into a buffer (a list, because it gets appended all the time). So setting a list to None will definitely free the memory, right?
另外:有时我会切片矩阵,计算一些值并将值放入缓冲区(一个列表,因为它一直被附加)。因此,将列表设置为 None 肯定会释放内存,对吗?
Or does some kind of unset()
method exist where whole identifiers plus its referenced objects are "deleted"?
或者是否unset()
存在某种方法来“删除”整个标识符及其引用的对象?
采纳答案by innoSPG
You definitely want to have a look at the garbage collection. Unlike some programming language like C/C++
where the programmer has to free dynamically allocated memory by himself when the space is no longer needed, python has a garbage collection. Meaning that python itself frees the memory when necessary
.
你肯定想看看垃圾收集。与某些编程语言不同,例如C/C++
当不再需要空间时程序员必须自己释放动态分配的内存,python 具有垃圾收集功能。这意味着 python 本身在necessary
.
When you use some_matrix = None
, you unlink the variable from the memory space; the reference counter is decreased, and if it reaches 0
, the garbage collector will free the memory.
When you use del some_matrix
as suggested by MSeifert, the memory is not freed immediately as opposed to what the answer says. According to python doc, this is what happens:
当您使用 时some_matrix = None
,您将变量与内存空间解除链接;引用计数器减少,如果达到0
,垃圾收集器将释放内存。当您del some_matrix
按照 MSeifert 的建议使用时,与答案所说的相反,内存不会立即释放。根据python doc,这是发生了什么:
Deletion of a name removes the binding of that name from the local or global namespace
删除名称会从本地或全局名称空间中删除该名称的绑定
What happened under the hood is that the counter of references to the memory space is reduced by 1
independently of assigning None
or using del
. When this counter reaches 0
, the garbage collector will free
the memory space in the future. The only difference is that when using del
, it is clear from the context that you do not need the name anymore.
在幕后发生的事情是,通过1
独立于分配None
或使用del
. 当这个计数器达到 时0
,垃圾收集器将free
在未来占用内存空间。唯一的区别是,当使用 时del
,从上下文中可以清楚地看出您不再需要该名称。
If you look at the doc of the garbage collection, you will see that you can invoke it by yourself or change some of its parameters.
如果您查看垃圾收集的文档,您会发现您可以自己调用它或更改它的一些参数。
回答by MSeifert
Numpy
deletes arrays when the reference counter is zero (or at least it keeps track of the reference counter and let's the OS collect the garbage).
Numpy
当引用计数器为零时删除数组(或者至少它跟踪引用计数器并让操作系统收集垃圾)。
For example having
例如有
import numpy as np
a = np.linspace(0,100, 10000000)
a = None
will free the memory "immediatly" (preferred way is writing del a
though) while
将“立即”释放内存(del a
尽管首选方式是写入),而
import numpy as np
a = np.linspace(0,100, 10000000)
b = a
a = None
will free nothing.
不会释放任何东西。
You mentioned also slicing. Slicing is just a view on the data and therefore exactly like the second example. If you don't delete both variables that reference the same array the OS will keep the arrays.
你也提到了切片。切片只是对数据的一个视图,因此与第二个示例完全相同。如果您不删除引用同一数组的两个变量,操作系统将保留这些数组。
If I do something very memory expensive I'll always stick with seperate functions that do the operation and only return what is really necessary. Functions clean up after themselves so any intermediate results are freed (If they are not returned).
如果我做了一些非常昂贵的内存操作,我将始终坚持使用单独的函数来执行操作并且只返回真正需要的东西。函数会自行清理,以便释放任何中间结果(如果未返回它们)。
回答by Amir
In case you have to do something like below memory won't be freed although a copy of a
will be made implicitly:
如果您必须执行以下操作,则不会释放内存,尽管a
会隐式制作的副本:
a = np.ones((10000, 10000))
b = np.empty((10000, 10000))
b[:] = a
a = None
del a
Instead you can do the following and memory will be freed after doing a = None
:
相反,您可以执行以下操作,并在执行后释放内存a = None
:
a = np.ones((10000, 10000))
b = np.empty((10000, 10000))
b[:] = np.copy(a)
a = None
del a