Python/Numpy 内存错误
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/4318615/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Python/Numpy MemoryError
提问by tylerthemiler
Basically, I am getting a memory error in python when trying to perform an algebraic operation on a numpy matrix. The variable u, is a large matrix of double (in the failing case its a 288x288x156 matrix of doubles. I only get this error in this huge case, but I am able to do this on other large matrices, just not this big). Here is the Python error:
基本上，当我尝试对 numpy 矩阵执行代数运算时，我在 python 中遇到内存错误。变量u, 是一个大的 double 矩阵（在失败的情况下，它是一个 288x288x156 的 double 矩阵。我只在这个巨大的案例中得到这个错误，但我能够在其他大矩阵上做到这一点，只是没有这么大）。这是Python错误：
Traceback (most recent call last):
File "S:D_Simulation_Data\Patient SPM Segmentation pc
t perim erosion flattop\SwSim.py", line 121, in __init__
self.mainSimLoop()
File "S:D_Simulation_Data\Patient SPM Segmentation pc
t perim erosion flattop\SwSim.py", line 309, in mainSimLoop
u = solver.solve_cg(u,b,tensors,param,fdHold,resid) # Solve the left hand si
de of the equation Au=b with conjugate gradient method to approximate u
File "S:D_Simulation_Data\Patient SPM Segmentation pc
t perim erosion flattop\conjugate_getb.py", line 47, in solv
e_cg
u = u + alpha*p
MemoryError
u = u + alpha*pis the line of code that fails.
u = u + alpha*p是失败的代码行。
alphais just a double, while uand rare the large matrices described above (both of the same size).
alpha只是一个双精度型，而u和r是上述的大矩阵（两者大小相同）。
I don't know that much about memory errors especially in Python. Any insight/tips into solving this would be very appreciated!
我不太了解内存错误，尤其是在 Python 中。任何解决此问题的见解/提示将不胜感激！
Thanks
谢谢
采纳答案by luispedro
Rewrite to
改写为
p *= alpha
u += p
and this will use much less memory. Whereas p = p*alphaallocates a whole new matrix for the result of p*alphaand then discards the old p; p*= alphadoes the same thing in place.
这将使用更少的内存。而p = p*alpha为结果分配一个全新的矩阵p*alpha，然后丢弃旧的p；p*= alpha就地做同样的事情。
In general, with big matrices, try to use op=assignment.
一般来说，对于大矩阵，尽量使用op=赋值。
回答by Benjamin Bannier
Your matrix has 288x288x156=12,939,264 entries, which for doublecould come out to 400MB in memory. numpythrowing a MemoryErrorat you just means that in the function you called the memory needed to perform the operation wasn't available from the OS.
您的矩阵有 288x288x156=12,939,264 个条目，double内存中可能会出现 400MB。numpy向MemoryError您抛出 a仅意味着在您调用的函数中，操作系统无法提供执行操作所需的内存。
If you can work with sparse matrices this might save you a lot of memory.
如果您可以使用稀疏矩阵，这可能会为您节省大量内存。
回答by DaveP
Another tip I have found to avoid memory errors is to manually control garbage collection. When objects are deleted or go our of scope, the memory used for these variables isn't freed up until a garbage collection is performed. I have found with some of my code using large numpy arrays that I get a MemoryError, but that I can avoid this if I insert calls to gc.collect() at appropriate places.
我发现的另一个避免内存错误的技巧是手动控制垃圾收集。当对象被删除或进入我们的作用域时，在执行垃圾收集之前不会释放用于这些变量的内存。我发现我的一些使用大型 numpy 数组的代码出现了 MemoryError，但是如果我在适当的位置插入对 gc.collect() 的调用，我可以避免这种情况。
You should only look into this option if using "op=" style operators etc doesn't solve your problem as it's probably not the best coding practice to have gc.collect() calls everywhere.
如果使用“op=”样式运算符等不能解决您的问题，您应该只查看此选项，因为在任何地方都调用 gc.collect() 可能不是最佳编码实践。
