列出 ipython 和 jupyter 中的内存使用情况
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/40993626/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
list memory usage in ipython and jupyter
提问by Rho Phi
I have a few (almost ten) Gb of memory taken by the ipython kernel. I think this is coming from large objects (matrices, lists, numpy arrays, ...) that I might have produced during some operation and now I do not need anymore.
我有几个(几乎十个) Gb 的内存被 ipython 内核占用。我认为这是来自我可能在某些操作期间产生的大对象(矩阵、列表、numpy 数组,...),现在我不再需要了。
I would like to list all of the objects I have definedand sort themby their memory footprint. Is there a simple way to do that? For certain types there is nbytes
method, but not for all ... so I am looking for a general way to list all objects I have made and their memory occupation.
我想列出我定义的所有对象,并按它们的内存占用量对它们进行排序。有没有一种简单的方法可以做到这一点?对于某些类型有nbytes
方法,但不是所有的......所以我正在寻找一种通用的方法来列出我制作的所有对象及其内存占用。
回答by Abdou
Assuming that you are using ipython
or jupyter
, you will need to do a little bit of work to get a list all of the objects youhave defined. That means taking everything available in globals()
and filtering out objects that are modules
, builtins
, ipython objects
, etc. Once you are sure you have those objects, then you can proceed to grabbing their sizes with sys.getsizeof
. This can be summed up as follows:
假设您正在使用ipython
或jupyter
,您需要做一些工作来获取您定义的所有对象的列表。这意味着要提供一切globals()
和剔除这些对象modules
,builtins
,ipython objects
,等等。一旦你确定你有这些对象,那么你就可以继续与抢夺它们的大小sys.getsizeof
。这可以总结如下:
import sys
# These are the usual ipython objects, including this one you are creating
ipython_vars = ['In', 'Out', 'exit', 'quit', 'get_ipython', 'ipython_vars']
# Get a sorted list of the objects and their sizes
sorted([(x, sys.getsizeof(globals().get(x))) for x in dir() if not x.startswith('_') and x not in sys.modules and x not in ipython_vars], key=lambda x: x[1], reverse=True)
Please keep in mind that for python objects (those created with python's builtin functions), sys.getsizeof
will be very accurate. But it can be a bit inaccurate on objects created using third-party libraries. Furthermore, please be mindful that sys.getsizeof
adds an additional garbage collector overhead if the object is managed by the garbage collector. So, some things may look a bit heavier than they actually are.
请记住,对于 python 对象(使用 python 的内置函数创建的对象),sys.getsizeof
将非常准确。但是对于使用第三方库创建的对象,它可能有点不准确。此外,请注意,sys.getsizeof
如果对象由垃圾收集器管理,则会增加额外的垃圾收集器开销。所以,有些东西看起来可能比实际重一些。
As a side note, numpy
's .nbytes
method can be somewhat misleading in that it does not include memory consumed by non-element attributes of the array object.
作为旁注,numpy
的.nbytes
方法可能有些误导,因为它不包括数组对象的非元素属性消耗的内存。
I hope this helps.
我希望这有帮助。