如何在python中使用valgrind?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/20112989/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-18 19:35:36  来源:igfitidea点击:

How to use valgrind with python?

pythonmemory-leaksvalgrind

提问by Dejan Jovanovi?

I am trying to memcheck a C python extension I am writing, but I'm having trouble setting up valgrind to work with python. I would really appreciate some advice. Just for context, this is Ubuntu 13.10, python 2.7.5+, and valgrind 3.8.1.

我正在尝试 memcheck 我正在编写的 C python 扩展,但是我在设置 valgrind 以使用 python 时遇到问题。我真的很感激一些建议。仅就上下文而言,这是 Ubuntu 13.10、python 2.7.5+ 和 valgrind 3.8.1。

As per recommendation from Readme.valgrindI did the following.

根据Readme.valgrind我的建议,我做了以下事情。

1) Downloaded the python source with

1)下载python源代码

sudo apt-get build-dep python2.7
apt-get source python2.7

2) Applied the code patch, i.e. "Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c".

2) 应用了代码补丁,即“在 Objects/obmalloc.c 中取消注释 Py_USING_MEMORY_DEBUGGER”。

3) Applied the suppression patch, i.e. "Uncomment the lines in Misc/valgrind-python.supp that suppress the warnings for PyObject_Free and PyObject_Realloc"

3) 应用了抑制补丁,即“取消注释 Misc/valgrind-python.supp 中抑制 PyObject_Free 和 PyObject_Realloc 警告的行”

4) Compiled python with

4)编译python

./configure --prefix=/home/dejan/workspace/python --without-pymalloc
make -j4 install

Note that I did both 2 and 3, while README.valgrind says to do 2 or 3... more can't hurt.

请注意,我做了 2 和 3,而 README.valgrind 说要做 2 或 3 ......更多不会伤害。

Now, let's test this on some sample python code in test.py

现在,让我们在一些示例 Python 代码上对此进行测试 test.py

print "Test"

Let's run valgrind on python with this script

让我们用这个脚本在 python 上运行 valgrind

valgrind --tool=memcheck --leak-check=full --suppressions=python2.7-2.7.5/Misc/valgrind-python.supp bin/python test.py

Unexpectedly, there is still loads of reports from valgrind, with the first one being (and many more following)

出乎意料的是,仍然有大量来自 valgrind 的报告,第一个是(以及更多后续)

==27944== HEAP SUMMARY:
==27944==     in use at exit: 857,932 bytes in 5,144 blocks  
==27944==   total heap usage: 22,766 allocs, 17,622 frees, 4,276,934 bytes allocated
==27944== 
==27944== 38 bytes in 1 blocks are possibly lost in loss record 24 of 1,343
==27944==    at 0x4C2A2DB: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27944==    by 0x46B8DD: PyString_FromString (stringobject.c:143)
==27944==    by 0x439631: PyFile_FromFile (fileobject.c:157)
==27944==    by 0x4E9B4A: _PySys_Init (sysmodule.c:1383)
==27944==    by 0x4E29E9: Py_InitializeEx (pythonrun.c:222)
==27944==    by 0x4154B4: Py_Main (main.c:546)
==27944==    by 0x577DDE4: (below main) (libc-start.c:260)

Am I doing something wrong? Is there a way to valgrind a python script that doesn't leak and get clean valgrind output?

难道我做错了什么?有没有办法对不泄漏的python脚本进行valgrind并获得干净的valgrind输出?

采纳答案by Dejan Jovanovi?

I found the answer here.

我在这里找到了答案。

Python also needs to be compiled in debug mode, i.e.

Python也需要在调试模式下编译,即

./configure --prefix=/home/dejan/workspace/python --without-pymalloc --with-pydebug --with-valgrind

In addition, numpy has a suppresion filethat gets rid of the extra valgrind warnings.

此外,numpy 有一个消除额外 valgrind 警告的抑制文件

回答by dequis

Since python 3.6, there's a PYTHONMALLOCenvironment variable which is available in release builds, without needing to recompile.

从 python 3.6 开始,有一个PYTHONMALLOC环境变量可以在发布版本中使用,无需重新编译。

PYTHONMALLOC=malloc python3 foobar.py

This will disable pymalloc and just use the libc malloc directly, making it valgrind-friendly. This is equivalent to --without-pymalloc(and it is just as slow)

这将禁用 pymalloc 并直接使用 libc malloc,使其对 valgrind 友好。这等效于--without-pymalloc(并且同样慢)

If valgrind is too slow, other values can be helpful. PYTHONMALLOC=debugand PYTHONMALLOC=malloc_debugadd debug hooks on top of the default and the libc allocators respectively. Their effects, from the docs:

如果 valgrind 太慢,其他值可能会有所帮助。PYTHONMALLOC=debugPYTHONMALLOC=malloc_debug分别在默认分配器和 libc 分配器之上添加调试钩子。它们的效果,来自文档:

  • Newly allocated memory is filled with the byte 0xCB
  • Freed memory is filled with the byte 0xDB
  • Detect violations of the Python memory allocator API. For example, PyObject_Free() called on a memory block allocated by PyMem_Malloc().
  • Detect writes before the start of a buffer (buffer underflows)
  • Detect writes after the end of a buffer (buffer overflows)
  • Check that the GIL is held when allocator functions of PYMEM_DOMAIN_OBJ (ex: PyObject_Malloc()) and PYMEM_DOMAIN_MEM (ex: PyMem_Malloc()) domains are called.
  • 新分配的内存用字节 0xCB 填充
  • 释放的内存填充字节 0xDB
  • 检测违反 Python 内存分配器 API 的情况。例如, PyObject_Free() 调用 PyMem_Malloc() 分配的内存块。
  • 在缓冲区开始之前检测写入(缓冲区下溢)
  • 检测缓冲区结束后的写入(缓冲区溢出)
  • 当调用 PYMEM_DOMAIN_OBJ(例如:PyObject_Malloc())和 PYMEM_DOMAIN_MEM(例如:PyMem_Malloc())域的分配器函数时,检查 GIL 是否保持。

This will catch some uninitialized reads, some use after free, some buffer under/overflows, etc, but won't report leaks and won't touch memory that isn't allocated through python (When using glibc, the MALLOC_PERTURB_and MALLOC_CHECK_environment variables might help there)

这将捕获一些未初始化的读取、一些释放后使用、一些缓冲区下/溢出等,但不会报告泄漏,也不会触及未通过 python 分配的内存(使用 glibc 时,环境变量MALLOC_PERTURB_MALLOC_CHECK_环境变量可能会有所帮助那里)

See also:

也可以看看: