C语言 C 中的 malloc 可以有多大？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/3463207/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How big can a malloc be in C?
提问by Derek
I have a malloc in C that is 26901^2*sizeof(double)
我在 C 中有一个 malloc，它是 26901^2*sizeof(double)
This got me thinking what the largest value can be here?
这让我想到这里的最大值是多少？
Also, would I have any problems defining a macro to access this 2D array?
另外，定义一个宏来访问这个二维数组会有什么问题吗？
#define DN(i,j) ((int)i * ny + (int)j)
Because this seems to not be working for me - or I am at least unsure it is. I can't figure out how to make totalview dive on a macro to tell me what A[DN(indx,jndx)] is actually looking at.
因为这似乎对我不起作用 - 或者我至少不确定它是。我不知道如何在宏上进行 totalview 潜水来告诉我 A[DN(indx,jndx)] 实际上在看什么。
回答by Matt Joiner
Observations
观察
Assuming a typical allocator, such as the one glibc uses, there are some observations:
假设一个典型的分配器，例如 glibc 使用的一个，有一些观察结果：
- Whether or not the memory is actually used, the region must be reserved contiguously in virtual memory.
- The largest free contiguous regions depends on the memory usage of existing memory regions, and the availability of those regions to
malloc. - The mapping practices depend on the architecture and OS. Furthermore underlying system calls to obtain memory regions are affected by these practices (such as
malloccalling through tommapto acquire pages).
- 无论内存是否实际使用，该区域都必须在虚拟内存中连续保留。
- 最大的空闲连续区域取决于现有内存区域的内存使用情况，以及这些区域的可用性
malloc。 - 映射实践取决于体系结构和操作系统。此外，获取内存区域的底层系统调用会受到这些做法的影响（例如
malloc调用 tommap来获取页面）。
Experiment
实验
Here's a simple programto allocate the largest possible block (compile with gcc largest_malloc_size.c -Wall -O2:
这是一个分配最大可能块的简单程序（编译时使用gcc largest_malloc_size.c -Wall -O2：
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static void *malloc_wrap(size_t size)
{
void *p = malloc(size);
if (p) {
printf("Allocated %zu bytes from %p to %p\n", size, p, p + size);
}
else {
printf("Failed to allocated %zu bytes\n", size);
}
return p;
}
int main()
{
size_t step = 0x1000000;
size_t size = step;
size_t best = 0;
while (step > 0)
{
void *p = malloc_wrap(size);
if (p) {
free(p);
best = size;
}
else {
step /= 0x10;
}
size += step;
}
void *p = malloc_wrap(best);
if (p) {
pause();
return 0;
}
else {
return 1;
}
}
Running the above program (./a.out) on my Linux stanley 2.6.32-24-generic-pae #39-Ubuntu SMP Wed Jul 28 07:39:26 UTC 2010 i686 GNU/Linuxmachine obtains this result:
./a.out在我的Linux stanley 2.6.32-24-generic-pae #39-Ubuntu SMP Wed Jul 28 07:39:26 UTC 2010 i686 GNU/Linux机器上运行上述程序 ( )得到以下结果：
<snip>
Allocated 2919235584 bytes from 0x9763008 to 0xb7763008
Allocated 2936012800 bytes from 0x8763008 to 0xb7763008
Failed to allocated 2952790016 bytes
Failed to allocated 2953838592 bytes
Failed to allocated 2953904128 bytes
Failed to allocated 2953908224 bytes
Allocated 2936012800 bytes from 0x85ff008 to 0xb75ff008
This is an allocation of exactly 2800MiB. Observing the relevant mapping from /proc/[number]/maps:
这是恰好 2800MiB 的分配。从/proc/[number]/maps以下观察相关映射：
<snip>
0804a000-0804b000 rw-p 00001000 08:07 3413394 /home/matt/anacrolix/public/stackoverflow/a.out
085ff000-b7600000 rw-p 00000000 00:00 0 [heap]
b7600000-b7621000 rw-p 00000000 00:00 0
b7621000-b7700000 ---p 00000000 00:00 0
b7764000-b7765000 rw-p 00000000 00:00 0
b7765000-b78b8000 r-xp 00000000 08:08 916041 /lib/tls/i686/cmov/libc-2.11.1.so
<snip>
bfc07000-bfc1c000 rw-p 00000000 00:00 0 [stack]
Conclusion
结论
It appears the heap has been expanded in the area between the program data and code, and the shared library mappings, which sit snug against the user/kernel memory space boundary(obviously 3G/1G on this system).
看起来堆已经在程序数据和代码以及共享库映射之间的区域中扩展，它们紧贴用户/内核内存空间边界（在该系统上显然是 3G/1G）。
This result suggests that the maximum allocatable space using malloc is roughly equal to:
这个结果表明使用 malloc 的最大可分配空间大致等于：
- The user space region (3GB in the example)
- Less the offset to the start of the heap (program code and data)
- Less the space reserved for the main thread stack
- Less the space occupied by all the mapped in shared libraries
- Finally, the largest contiguous region that can be found by the underlying system call within the region available to the heap (which may be fragmented by other mappings)
- 用户空间区域（示例中为 3GB）
- 减去堆开始的偏移量（程序代码和数据）
- 减少为主线程栈保留的空间
- 减少共享库中所有映射所占用的空间
- 最后，在堆可用区域内，底层系统调用可以找到的最大连续区域（可能被其他映射分割）
Notes
笔记
With respect to glibc and Linux implementations, the following manual snippets are of great interest:
关于 glibc 和 Linux 实现，以下手动片段非常有趣：
Normally, malloc() allocates memory from the heap, and adjusts the size
of the heap as required, using sbrk(2). When allocating blocks of mem‐
ory larger than MMAP_THRESHOLD bytes, the glibc malloc() implementation
allocates the memory as a private anonymous mapping using mmap(2).
MMAP_THRESHOLD is 128 kB by default, but is adjustable using mal‐
lopt(3).
MAP_ANONYMOUS
The mapping is not backed by any file; its contents are initial‐
ized to zero.
Afterword
后记
This test was done on a x86 kernel. I'd expect similar results from a x86_64 kernel, albeit with vastly larger memory regions returned. Other operating systems may vary in their placement of mappings, and the handling of large mallocs, so results could be quite considerably different.
此测试是在 x86 内核上完成的。我希望 x86_64 内核有类似的结果，尽管返回的内存区域要大得多。其他操作系统在映射的位置和大mallocs的处理方面可能会有所不同，因此结果可能会有很大不同。
回答by Hut8
That depends on your malloc implementation!
这取决于您的 malloc 实现！
According to Wikipedia, "Since the v2.3 release, the GNU C library (glibc) uses a modified ptmalloc2, which itself is based on dlmalloc v2.7.0." dlmalloc refers to Doug Lea's malloc implementation. The important thing to note in this implementation is that large mallocs are accomplished through the operating system's memory mapped file functionality, so these blocks can be quite large indeed without many problems of finding a contiguous block.
根据维基百科，“自 v2.3 发布以来，GNU C 库 (glibc) 使用修改后的 ptmalloc2，它本身基于 dlmalloc v2.7.0。” dlmalloc 指的是 Doug Lea 的 malloc 实现。在这个实现中需要注意的重要一点是，大型 malloc 是通过操作系统的内存映射文件功能完成的，因此这些块确实可以非常大，而不会遇到很多寻找连续块的问题。
回答by mvds
The malloc question is answered (depends on OS, which you don't specify), so about that define:
malloc 问题已回答（取决于您未指定的操作系统），因此定义：
#define DN(i,j) ((int)i * ny + (int)j)
is not quite safe, for someone might do DN(a+b,c)which expands to
不太安全，因为有人可能会DN(a+b,c)扩展到
((int)a+b * ny + (int)c)
which is probably not what you wanted. So put a lot of parentheses in there:
这可能不是您想要的。所以在那里放了很多括号：
#define DN(i,j) ((int)(i) * ny + (int)(j))
to see what DN(indx,jndx)points to, just printf("%d\n",DN(indx,jndx));
看什么DN(indx,jndx)指向，只是printf("%d\n",DN(indx,jndx));
回答by Dummy00001
This got me thinking what the largest value can be here?
这让我想到这里的最大值是多少？
26'901^2 = 723'663'801. If your double is 8 bytes, then it is less than 8GB. I see totally no problem allocating that much of memory and my apps routinely allocate (on 64 bit systems) much more. (Biggest memory consumption I have ever seen was 420GB (on Solaris 10 numa system with 640GB RAM) with largest continuous block of ~24GB.)
26'901^2 = 723'663'801。如果您的 double 是 8 个字节，则它小于 8GB。我认为分配那么多内存完全没有问题，我的应用程序通常会分配（在 64 位系统上）更多。（我见过的最大内存消耗是 420GB（在具有 640GB RAM 的 Solaris 10 numa 系统上），最大连续块约为 24GB。）
Largest value is hard to identify since it is platform dependent: similar to the 32bit systems it depends on user-space / kernel-space split. As things stand at the moment, I think one would first come to the limit of the actual physical RAM - before reaching the limit of what libc can allocate. (And kernel doesn't care, it just expands virtual memory often without even considering whether there is sufficient RAM to pin it to.)
最大值很难确定，因为它依赖于平台：类似于 32 位系统，它依赖于用户空间/内核空间分割。就目前的情况而言，我认为人们首先会达到实际物理 RAM 的限制 - 在达到 libc 可以分配的限制之前。（内核并不关心，它只是经常扩展虚拟内存，甚至不考虑是否有足够的 RAM 来固定它。）
回答by caf
The largest memory block you can askmalloc()for is the largest size_tvalue - this is SIZE_MAXfrom <limits.h>. The largest amount you can sucessfullyrequest is obviously dependent on the operating system and the configuration of the individual machine.
您可以要求的最大内存块malloc()是size_t最大值 - 这是SIZE_MAX来自<limits.h>. 您可以成功请求的最大数量显然取决于操作系统和单个机器的配置。
Your macro is not safe. It performs the index calculation with an intvariable, which is only required to have a range up to 32767. Any value higher than this can cause signed overflow, which results in undefined behaviour. You are probably best off doing the calculation as a size_t, since that type must be able to hold any valid array index:
你的宏不安全。它使用一个int变量执行索引计算，该变量的范围只需要达到 32767。任何高于此值的值都可能导致有符号溢出，从而导致未定义的行为。您最好将计算作为 a 进行size_t，因为该类型必须能够保存任何有效的数组索引：
#define DN(i, j) ((size_t)(i) * ny + (size_t)(j))
(Although note that if you supply negative values for ior j, you'll get an index far out of bounds).
（但请注意，如果您为ior提供负值j，您将得到一个远远超出范围的索引）。
回答by Will
The size parameter in a call to malloc is of type size_t, which varies by implementation. See this questionfor more.
malloc 调用中的 size 参数是 size_t 类型，它因实现而异。请参阅此问题了解更多信息。
