Laravel / Eloquent 内存泄漏重复检索相同的记录

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/18775510/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-14 08:25:04  来源:igfitidea点击:

Laravel / Eloquent memory leak retrieving the same record repeatedly

laravellaravel-4

提问by Brian Hanson

I am trying to write a laravel function that gets lots of records (100,000+) from one database and puts it in another database. Towards that end, I need to query my database and see if the user already exists. I repeatedly call this code:

我正在尝试编写一个 laravel 函数,该函数从一个数据库中获取大量记录(100,000+)并将其放入另一个数据库中。为此,我需要查询我的数据库并查看用户是否已经存在。我反复调用这段代码:

$users = User::where('id', '=', 2)->first();

And then after that happens a few hundred times, I run out of memory. So, I made a minimalist example of it using up all the available memory, and it looks like this:

然后在这种情况发生了几百次之后,我的内存不足。所以,我做了一个使用所有可用内存的简约示例,它看起来像这样:

<?php

use Illuminate\Console\Command;

class memoryleak extends Command
{
    protected $name = 'command:memoryleak';
    protected $description = 'Demonstrates memory leak.';

    public function fire()
    {
        ini_set("memory_limit","12M");

        for ($i = 0; $i < 100000; $i++)
        {
            var_dump(memory_get_usage());
            $this->external_function();
        }
    }

    function external_function()
    {
        // Next line causes memory leak - comment out to compare to normal behavior
        $users = User::where('id', '=', 2)->first();

        unset($users);
        // User goes out of scope at the end of this function
    }
}

And the output of this script (executed by 'php artisan command:memoryleak') looks something like this:

这个脚本的输出(由 'php artisan command:memoryleak' 执行)看起来像这样:

int(9298696)
int(9299816)
int(9300936)
int(9302048)
int(9303224)
int(9304368)
....
int(10927344)
int(10928432)
int(10929560)
int(10930664)
int(10931752)
int(10932832)
int(10933936)
int(10935072)
int(10936184)
int(10937320)
....
int(12181872)
int(12182992)
int(12184080)
int(12185192)
int(12186312)
int(12187424)
PHP Fatal error:  Allowed memory size of 12582912 bytes exhausted (tried to allocate 89 bytes) in /Volumes/Mac OS/www/test/vendor/laravel/framework/src/Illuminate/Database/Connection.php on line 275

If I comment out the line "$users = User::where('id', '=', 2)->first();" then the memory usage stays stable.

如果我注释掉“$users = User::where('id', '=', 2)->first();” 那么内存使用量保持稳定。

Does anyone have any insight as to why this line would use memory like this, or know a smarter way to accomplish what I am trying to do?

有没有人知道为什么这条线会像这样使用内存,或者知道一种更聪明的方法来完成我想要做的事情?

Thank you for your time.

感谢您的时间。

回答by Erik

I recreated your script and stepped through it with a debugger because I couldn't fathom what sort of horrible thing would cause this type of memory issue. As I stepped through, I came across this:

我重新创建了您的脚本并使用调试器逐步完成了它,因为我无法理解什么样的可怕事情会导致这种类型的内存问题。当我走过去时,我遇到了这个:

// in Illuminate\Database\Connection
$this->queryLog[]?=?compact('query',?'bindings',?'time');

It seems every query you run in Laravel is stored in a persistent log, which explains your increasing memory usage after each query. Just above that, is the following line:

你在 Laravel 中运行的每个查询似乎都存储在一个持久日志中,这解释了每次查询后内存使用量的增加。就在这之上,是以下几行:

if ( ! $this->loggingQueries) return;

A little more digging determined that the loggingQueriesproperty is set to true by default, and can be changed via the disableQueryLogmethod, so that means, if you call:

进一步挖掘确定该loggingQueries属性默认设置为 true,并且可以通过该disableQueryLog方法进行更改,因此这意味着,如果您调用:

 DB::connection()->disableQueryLog();

before you're going to execute all your queries, you won't see ever increasing memory usage; it solved the problem when I ran my test based on your example code. When you're done, if you don't want to affect the rest of the application you could call

在执行所有查询之前,您不会看到内存使用量不断增加;当我根据您的示例代码运行测试时,它解决了问题。完成后,如果您不想影响应用程序的其余部分,则可以调用

DB::connection()->enableQueryLog();

to renable logging.

重新启用日志记录。

回答by Shauna

I can't say why it isn't releasing memory. Your best bet is to follow the code and learn how it does what it does for that one. Or ask Taylor.

我不能说为什么它不释放内存。最好的办法是遵循代码并了解它是如何为那个代码做的。或者问泰勒。

As for other things you can do:

至于你可以做的其他事情:

Cache the queryIf you're calling the same query over and over and over, then use the query cache. It's as simple as adding ->remember($time_to_cache)to your query.

缓存查询如果您一遍又一遍地调用相同的查询,请使用查询缓存。它就像添加->remember($time_to_cache)到您的查询一样简单。

Make the DBMS do all the hard work.Ideally, you'd just do an insert into selectstatement, but that gets hairy when you're crossing databases. In lieu of that, batch both the select and the insert queriesso that you're making fewer calls to the databases and creating fewer objects. This offloads more of the heavy lifting to the database management system, which is arguably more efficient at these types of tasks.

让 DBMS 完成所有繁重的工作。理想情况下,您只需执行一个insert into selectstatement,但是当您跨数据库时,这会变得很麻烦。取而代之的是,批处理选择和插入查询,以便您对数据库进行更少的调用并创建更少的对象。这将更多繁重的工作交给了数据库管理系统,可以说在这些类型的任务中效率更高。