php PHP数组中的内存优化

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/6336528/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-26 00:04:04  来源:igfitidea点击:

Memory optimization in PHP array

phparraysmemory-management

提问by Gabriel A. Zorrilla

I'm working with a large array which is a height map, 1024x1024 and of course, i'm stuck with the memory limit. In my test machine i can increase the mem limit to 1gb if i want, but in my tiny VPS with only 256 ram, it's not an option.

我正在使用一个大数组,它是一个高度图,1024x1024,当然,我遇到了内存限制。在我的测试机器上,如果我愿意,我可以将内存限制增加到 1GB,但是在我只有 256 ram 的小型 VPS 中,这不是一个选择。

I've been searching in stack and google and found several "well, you are using PHP not because memory efficiency, ditch it and rewrite in c++" and honestly, that's ok and I recognize PHP loves memory.

我一直在堆栈和谷歌中搜索,发现了几个“好吧,你使用 PHP 不是因为内存效率,放弃它并用 C++ 重写”,老实说,这没关系,我认识到 PHP 喜欢内存。

But, when digging more inside PHP memory management, I did not find what memory consumes every data type. Or if casting to another type of data reduces mem consumption.

但是,当深入研究 PHP 内存管理时,我没有发现每种数据类型消耗的内存。或者如果转换为另一种类型的数据会减少内存消耗。

The only "optimization" technique i found was to unset variables and arrays, that's it.

我发现的唯一“优化”技术是取消设置变量和数组,就是这样。

Converting the code to c++ using some PHP parsers would solve the problem?

使用一些 PHP 解析器将代码转换为 C++ 会解决问题吗?

Thanks!

谢谢!

回答by Matthew

If you want a real indexed array, use SplFixedArray. It uses less memory. Also, PHP 5.3 has a much better garbage collector.

如果您想要一个真正的索引数组,请使用SplFixedArray。它使用更少的内存。此外,PHP 5.3 有一个更好的垃圾收集器。

Other than that, well, PHP will use more memory than a more carefully written C/C++ equivalent.

除此之外,PHP 将使用比更仔细编写的 C/C++ 等价物更多的内存。

Memory Usage for 1024x1024 integer array:

1024x1024 整数数组的内存使用:

  • Standard array: 218,756,848
  • SplFixedArray: 92,914,208
  • 标准阵列:218,756,848
  • SplFixedArray:92,914,208

as measured by memory_get_peak_usage()

memory_get_peak_usage()

$array = new SplFixedArray(1024 * 1024); // array();
for ($i = 0; $i < 1024 * 1024; ++$i)
  $array[$i] = 0;

echo memory_get_peak_usage();

Note that the same array in C using 64-bit integers would be 8M.

请注意,在 C 中使用 64 位整数的相同数组将为 8M。

As others have suggested, you could pack the data into a string. This is slower but muchmore memory efficient. If using 8 bit values it's super easy:

正如其他人所建议的那样,您可以将数据打包成一个字符串。这会慢一些,但很多更多的内存效率。如果使用 8 位值,则非常简单:

$x = str_repeat(chr(0), 1024*1024);
$x[$i] = chr($v & 0xff); // store value $v into $x[$i]
$v = ord($x[$i]);        // get value $v from $x[$i]

Here the memory will only be about 1.5MB (that is, when considering the entire overhead of PHP with just this integer string array).

这里的内存只有 1.5MB 左右(也就是说,考虑到 PHP 仅使用这个整数字符串数组的整个开销)。

For the fun of it, I created a simple benchmark of creating 1024x1024 8-bit integers and then looping through them once. The packed versions all used ArrayAccessso that the user code looked the same.

为了好玩,我创建了一个简单的基准测试,创建 1024x1024 8 位整数,然后循环遍历它们一次。全部使用打包版本ArrayAccess,因此用户代码看起来相同。

                   mem    write   read
array              218M   0.589s  0.176s
packed array       32.7M  1.85s   1.13s
packed spl array   13.8M  1.91s   1.18s
packed string      1.72M  1.11s   1.08s

The packed arrays used native 64-bit integers (only packing 7 bytes to avoid dealing with signed data) and the packed string used ordand chr. Obviously implementation details and computer specs will affect things a bit, but I would expect you to get similar results.

打包数组使用本机 64 位整数(仅打包 7 个字节以避免处理有符号数据),打包字符串使用ordchr。显然实现细节和计算机规格会影响一些事情,但我希望你能得到类似的结果。

So while the array was 6x faster it also used 125x the memory as the next best alternative: packed strings. Obviously the speed is irrelevant if you are running out of memory. (When I used packed strings directly without an ArrayAccessclass they were only 3x slower than native arrays.)

因此,虽然数组快了 6 倍,但它也使用了 125 倍的内存作为下一个最佳选择:打包字符串。显然,如果内存不足,速度无关紧要。(当我在没有ArrayAccess类的情况下直接使用打包字符串时,它们仅比本地数组慢 3 倍。)

In short, to summarize, I would use something other than pure PHP to process this data if speed is of any concern.

总之,总而言之,如果速度有任何问题,我会使用纯 PHP 以外的其他东西来处理这些数据。

回答by N.B.

In addition to the accepted answer and suggestions in the comments, I'd like to suggest PHP Judy array implementation.

除了评论中接受的答案和建议之外,我还想建议PHP Judy array implementation

Quick tests showed interesting results. An array with 1 million entries using regular PHP array data structure takes ~200 MB. SplFixedArray uses around 90 megabytes. Judy uses 8 megs. Tradeoff is in performance, Judy takes about double the time of regular php array implementation.

快速测试显示了有趣的结果。使用常规 PHP 数组数据结构的具有 100 万个条目的数组需要约 200 MB。SplFixedArray 使用大约 90 兆字节。朱迪使用 8 兆。权衡是性能,Judy 花费的时间大约是常规 php 数组实现的两倍。

回答by Marco

A little bit late to the party, but if you have a multidimensional array you can save a lot of RAM when you store the complete array as json.

参加聚会有点晚了,但是如果您有一个多维数组,则在将完整数组存储为 json 时可以节省大量 RAM。

$array = [];

$data = [];
$data["a"] = "hello";
$data["b"] = "world";

To store this array just use:

要存储此数组,只需使用:

$array[] = json_encode($data);

instead of

代替

$array[] = $data;

If you want to get the arrry back, just use something like:

如果你想得到 arrry 回来,只需使用类似的东西:

$myData = json_decode($array[0], true);

I had a big array with 275.000 sets and saved about 36% RAM consumption.

我有一个包含 275.000 组的大数组,节省了大约 36% 的 RAM 消耗。

EDIT: I found a more better way, when you zip the json string:

编辑:我找到了一个更好的方法,当你压缩 json 字符串时:

$array[] = gzencode(json_encode($data));

and unzip it when you need it:

并在需要时解压缩:

$myData = json_decode(gzdecode($array[0], true));

This saved me nearly 75% of RAM peak usage.

这为我节省了近 75% 的 RAM 峰值使用量。