如何在 Linux 上检查进程的堆大小

How to check heap size for a process on Linux

提问by bashrc

I was writing some code and it kept crashing. Later after digging the dumps I realized I was overshooting the maximum heap limit (life would have been easier if I had added a check on malloc). Although I fixed that, is there any way to increase my heap size?

我正在编写一些代码，但它一直崩溃。后来在挖掘转储之后，我意识到我超出了最大堆限制（如果我在 malloc 上添加了一个检查，生活会更容易）。虽然我解决了这个问题，但有没有办法增加我的堆大小？

PS: A quite similar questionhere but the reply is unclear to me.

PS：这里有一个非常相似的问题，但我的回答不清楚。

采纳答案by RedComet

The heap usually is as large as the addressable virtual memory on your architecture.

堆通常与架构上的可寻址虚拟内存一样大。

You should check your systems current limits with the ulimit -acommand and seek this line max memory size (kbytes, -m) 3008828, this line on my OpenSuse 11.4 x86_64 with ~3.5 GiB of ram says I have roughly 3GB of ram per process.

您应该使用ulimit -a命令检查系统当前限制并查找此行 max memory size (kbytes, -m) 3008828，我的 OpenSuse 11.4 x86_64 上的此行具有 ~3.5 GiB 的 ram 表示我每个进程大约有 3GB 的 ram。

Then you can truly test your system using this simple program to check max usable memory per process:

然后，您可以使用这个简单的程序来真正测试您的系统，以检查每个进程的最大可用内存：

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc,char* argv[]){
        size_t oneHundredMiB=100*1048576;
        size_t maxMemMiB=0;
        void *memPointer = NULL;
        do{
                if(memPointer != NULL){
                        printf("Max Tested Memory = %zi\n",maxMemMiB);
                        memset(memPointer,0,maxMemMiB);
                        free(memPointer);
                }
                maxMemMiB+=oneHundredMiB;
                memPointer=malloc(maxMemMiB);
        }while(memPointer != NULL);
        printf("Max Usable Memory aprox = %zi\n",maxMemMiB-oneHundredMiB);
        return 0;
}

This programs gets memory on 100MiB increments, presents the currently allocated memory, allocates 0's on it,then frees the memory. When the system can't give more memory, returns NULL and it displays the final max usable amount of ram.

该程序以 100MiB 的增量获取内存，呈现当前分配的内存，在其上分配 0，然后释放内存。当系统不能提供更多内存时，返回 NULL 并显示最终的最大可用内存量。

The Caveat is that your system will start to heavily swap memory in the final stages. Depending on your system configuration, the kernel might decide to kill some processes. I use a 100 MiB increments so there is some breathing space for some apps and the system. You should close anything that you don't want crashing.

需要注意的是，您的系统将在最后阶段开始大量交换内存。根据您的系统配置，内核可能会决定终止某些进程。我使用 100 MiB 增量，因此某些应用程序和系统有一些喘息空间。你应该关闭任何你不想崩溃的东西。

That being said. In my system where I'm writing this nothing crashed. And the program above reports barely the same as ulimit -a. The difference is that it actually tested the memory and by means of memset()confirmed the memory was given and used.

话虽如此。在我写这篇文章的系统中，没有任何崩溃。上面的程序报告几乎与ulimit -a. 不同的是，它实际测试了内存，并通过memset()确认内存给出和使用。

For comparison on a Ubuntu 10.04x86 VM with 256 MiB of ram and 400MiB of swap the ulimit report was memory size (kbytes, -m) unlimitedand my little program reported 524.288.000 bytes, which is roughly the combined ram and swap, discounting ram used by others software and the kernel.

为了在具有 256 MiB ram 和 400MiB 交换区的 Ubuntu 10.04x86 VM 上进行比较，ulimit 报告是 memory size (kbytes, -m) unlimited，我的小程序报告了 524.288.000 字节，这大致是 ram 和交换区的组合，不考虑其他软件和内核使用的 ram。

Edit: As Adam Zalcman wrote, ulimit -mis no longer honored on newer 2.6 and up linux kernels, so i stand corrected. But ulimit -vis honored. For practical results you should replace -m with -v, and look for virtual memory (kbytes, -v) 4515440. It seems mere chance that my suse box had the -m value coinciding with what my little utility reported. You should remember that this is virtual memory assigned by the kernel, if physical ram is insufficient it will take swap space to make up for it.

编辑：正如 Adam Zalcman 所写的那样，ulimit -m在较新的 2.6 及更高版本的 linux 内核上不再受到尊重，所以我更正了。但是ulimit -v很荣幸。对于实际结果，您应该将 -m 替换为 -v，然后查找virtual memory (kbytes, -v) 4515440. 我的 suse 盒子的 -m 值与我的小实用程序报告的值一致，这似乎只是偶然。你应该记住，这是内核分配的虚拟内存，如果物理内存不足，它将需要交换空间来弥补。

If you want to know how much physical ram is available without disturbing any process or the system, you can use

如果你想在不干扰任何进程或系统的情况下知道有多少物理内存可用，你可以使用

long total_available_ram =sysconf(_SC_AVPHYS_PAGES) * sysconf(_SC_PAGESIZE) ;

long total_available_ram =sysconf(_SC_AVPHYS_PAGES) * sysconf(_SC_PAGESIZE) ;

this will exclude cache and buffer memory, so this number can be far smaller than the actual available memory. OS caches can be quiet large and their eviction can give the needed extra memory, but that is handled by the kernel.

这将排除缓存和缓冲内存，因此该数字可能远小于实际可用内存。操作系统缓存可能非常大，它们的驱逐可以提供所需的额外内存，但这是由内核处理的。

回答by Cratylus

I think your original problem was that mallocfailed to allocate the requested memory on your system.

我认为您最初的问题是malloc未能在您的系统上分配请求的内存。

Why this happened is specific to your system.

发生这种情况的原因特定于您的系统。

When a process is loaded, it is allocated memory up to a certain address which is the system break point for the process. Beyond that address the memory is unmapped for the process. So when the process "hits" the "break" point it requests more memory from the system and one way to do this is via the system call sbrk
mallocwould do that under the hood but in your system for some reason it failed.

当一个进程被加载时，它被分配到某个地址的内存，该地址是该进程的系统断点。超出该地址的内存未映射到进程。因此，当进程“命中”“中断”点时，它会从系统请求更多内存，而一种方法是通过系统调用sbrk
malloc会在后台执行此操作，但在您的系统中由于某种原因它失败了。

There could be many reasons for this for example:
1) I think in Linux there is a limit for max memory size. I think it is ulimitand perhaps you hit that. Check if it is set to a limit
2) Perhaps your system was too loaded
3) Your program does bad memory management and you end up with fragemented memory so malloccan not get the chunk size you requested.
4) Your program corrupts the mallocinternal data structures i.e. bad pointer usage
etc

这可能有很多原因，例如：
1）我认为在 Linux 中存在最大内存大小的限制。我认为它是ulimit，也许你击中了。检查它是否设置为限制
2) 可能您的系统负载过大
3) 您的程序进行了糟糕的内存管理，最终导致内存碎片化，因此malloc无法获得您请求的块大小。
4）您的程序破坏了malloc内部数据结构，即错误的指针使用
等

回答by Adam Zalcman

Heap and memory management is a facility provided by your C library (likely glibc). It maintains the heap and returns chunks of memory to you every time you do a malloc(). It doesn't know heap size limit: every time you request more memory than what is available on the heap, it just goes and asks the kernel for more (either using sbrk()or mmap()).

堆和内存管理是由 C 库（可能是 glibc）提供的工具。每次执行malloc(). 它不知道堆大小限制：每次您请求的内存比堆上可用的内存多时，它只会向内核请求更多（使用sbrk()或mmap()）。

By default, kernel will almost always give you more memory when asked. This means that malloc()will always return a valid address. It's only when you refer to an allocated page for the first time that the kernel will actually bother to find a page for you. If it finds that it cannot hand you one it runs an OOM killer which according to certain measure called badness(which includes your process's and its children's virtual memory sizes, nice level, overall running time etc) selects a victim and sends it a SIGTERM. This memory management technique is called overcommit and is used by the kernel when /proc/sys/vm/overcommit_memoryis 0 or 1. See overcommit-accountingin kernel documentation for details.

默认情况下，内核几乎总是在询问时为您提供更多内存。这意味着malloc()将始终返回有效地址。只有当您第一次引用分配的页面时，内核才会真正为您查找页面。如果它发现它不能给你一个，它会运行一个 OOM 杀手，它根据某种称为badness 的度量（包括你的进程及其子进程的虚拟内存大小、nice 级别、总体运行时间等）选择一个受害者并向它发送一个SIGTERM. 这种内存管理技术称为 overcommit，当/proc/sys/vm/overcommit_memory为 0 或 1时由内核使用。有关详细信息，请参阅内核文档中的overcommit-accounting。

By writing 2 into /proc/sys/vm/overcommit_memoryyou can disable the overcommit. If you do that the kernel will actually check whether it has memory before promising it. This will result in malloc()returning NULL if no more memory is available.

通过将 2 写入/proc/sys/vm/overcommit_memory您可以禁用过度使用。如果你这样做，内核会在承诺之前检查它是否有内存。malloc()如果没有更多可用内存，这将导致返回 NULL。

You can also set a limit on the virtual memory a process can allocate with setrlimit()and RLIMIT_ASor with the ulimit -vcommand. Regardless of the overcommit setting described above, if the process tries to allocate more memory than the limit, kernel will refuse it and malloc()will return NULL. Note than in modern Linux kernel (including entire 2.6.x series) the limit on the resident size (setrlimit()with RLIMIT_RSSor ulimit -mcommand) is ineffective.

您还可以对进程可以使用setrlimit()和RLIMIT_AS或 使用ulimit -v命令分配的虚拟内存设置限制。不管上面描述的过量使用设置如何，如果进程尝试分配比限制更多的内存，内核将拒绝它并malloc()返回 NULL。请注意，在现代 Linux 内核（包括整个 2.6.x 系列）中，对驻留大小（setrlimit()withRLIMIT_RSS或ulimit -mcommand）的限制是无效的。

The session below was run on kernel 2.6.32 with 4GB RAM and 8GB swap.

下面的会话在具有 4GB RAM 和 8GB 交换的内核 2.6.32 上运行。

$ cat bigmem.c
#include <stdlib.h>
#include <stdio.h>

int main() {
  int i = 0;
  for (; i < 13*1024; i++) {
    void* p = malloc(1024*1024);
    if (p == NULL) {
      fprintf(stderr, "malloc() returned NULL on %dth request\n", i);
      return 1;
    }
  }
  printf("Allocated it all\n");
  return 0;
}
$ cc -o bigmem bigmem.c
$ cat /proc/sys/vm/overcommit_memory
0
$ ./bigmem
Allocated it all
$ sudo bash -c "echo 2 > /proc/sys/vm/overcommit_memory"
$ cat /proc/sys/vm/overcommit_memory
2
$ ./bigmem
malloc() returned NULL on 8519th request
$ sudo bash -c "echo 0 > /proc/sys/vm/overcommit_memory"
$ cat /proc/sys/vm/overcommit_memory
0
$ ./bigmem
Allocated it all
$ ulimit -v $(( 1024*1024 ))
$ ./bigmem
malloc() returned NULL on 1026th request
$

In the example above swapping or OOM kill could never occur, but this would change significantly if the process actually tried to touch all the memory allocated.

在上面的例子中，交换或 OOM 终止永远不会发生，但如果进程实际上试图接触所有分配的内存，这将发生显着变化。

To answer your question directly: unless you have virtual memory limit explicitly set with ulimit -vcommand, there is no heap size limit other than machine's physical resources or logical limit of your address space (relevant in 32-bit systems). Your glibc will keep allocating memory on the heap and will request more and more from the kernel as your heap grows. Eventually you may end up swapping badly if all physical memory is exhausted. Once the swap space is exhausted a random process will be killed by kernel's OOM killer.

直接回答您的问题：除非您使用ulimit -v命令明确设置了虚拟内存限制，否则除了机器的物理资源或地址空间的逻辑限制（与 32 位系统相关）之外，没有堆大小限制。您的 glibc 将继续在堆上分配内存，并随着堆的增长从内核请求越来越多的内存。最终，如果所有物理内存都用完，交换可能会很糟糕。一旦交换空间用完，一个随机进程将被内核的 OOM 杀手杀死。

Note however, that memory allocation may fail for many more reasons than lack of free memory, fragmentation or reaching a configured limit. The sbrk()and mmap()calls used by glib's allocator have their own failures, e.g. the program break reached another, already allocated address (e.g. shared memory or a page previously mapped with mmap()) or process's maximum number of memory mappings has been exceeded.

但是请注意，内存分配失败的原因可能不仅仅是缺少可用内存、碎片或达到配置的限制。glib 的分配器使用的sbrk()和mmap()调用有它们自己的失败，例如程序中断到达另一个，已经分配的地址（例如共享内存或先前映射的页面mmap()）或进程的最大内存映射数已超过。

回答by A Koscianski

I'd like to add one point to the previous answers.

我想在之前的答案中补充一点。

Apps have the illusion that malloc() returns 'solid' blocks; in reality, a buffer may exist scattered, pulverized, on many pages of RAM. The crucial fact here is this: the Virtual Memory of a process, containing its code or containing something as a large array, mustbe contiguous. Let's even admit that code and data be separated; a large array, char str[universe_size], must be contiguous.

应用程序会产生 malloc() 返回“实体”块的错觉；在现实中，缓冲区可能分散、粉碎地存在于 RAM 的许多页面上。这里的关键事实是：一个进程的虚拟内存，包含它的代码或包含一个大数组的东西，必须是连续的。我们甚至承认代码和数据是分开的；大数组 char str[universe_size] 必须是连续的。

Now: can a single app enlarge the heap arbitrarily, to alloc such an array?

现在：单个应用程序可以任意扩大堆，以分配这样的数组吗？

The answer could be 'yes' if there were nothing else running in the machine. The heap can be ridiculously huge, but it must have boundaries. At some point, calls to sbrk() (in Linux, the function that, in short, 'enlarges' the heap) should stumble on the area reserved for another application.

如果机器中没有其他东西在运行，答案可能是“是”。堆可以大得离谱，但它必须有边界。在某些时候，对 sbrk()（在 Linux 中，简而言之，“扩大”堆的函数）的调用应该会偶然发现为另一个应用程序保留的区域。

This linkprovides some interesting and clarifying examples, check it out. I did not find the info on Linux.

此链接提供了一些有趣且清晰的示例，请查看。我没有找到有关 Linux 的信息。