如何使用 Laravel 的块来避免内存不足?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/28082279/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How do I use Laravel's chunk to avoid running out of memory?
提问by Chris Farrugia
I am pulling around 100k records from temporary tables, doing some slight modification on the data, downloading photos, and then putting the fields I need to keep in a "master" table. This is quickly causing my app to crash as it runs out of memory.
我从临时表中提取了大约 10 万条记录,对数据进行了一些轻微修改,下载了照片,然后将我需要保留的字段放入“主”表中。这很快导致我的应用程序因内存不足而崩溃。
I read the very brief docs on using chunk() with Laravel's eloquent ORM but do not know how to even begin to implement it in my class.
我阅读了关于在 Laravel 雄辩的 ORM 中使用 chunk() 的非常简短的文档,但不知道如何开始在我的课堂上实现它。
Here is what I am currently doing:
这是我目前正在做的事情:
public function fire()
{
// Turn off query logging
DB::connection()->disableQueryLog();
$feeds = RetsFeed::where('active','=',1)->get();
foreach ($feeds as $feed)
{
$class = "TempListing{$feed->board}";
$listings = $class::orderBy('MatrixModifiedDT','desc')->get();
$listings->each(function($listing) use ($feed) {
ListingMigrator::migrateListing($listing,$feed);
echo "Feed: $feed->board\r\n";
echo "SubcondoName: $listing->SubCondoName\r\n";
echo "Development: $listing->Development\r\n";
echo "\r\n";
});
}
}
Each feed (or datasource) is dumped into a temp table in a different chore. That works fine. Then, I grab all of hte listings out of one table (which is around 30k on average) and run my ListingMigrator method.
每个提要(或数据源)都以不同的方式转储到临时表中。这很好用。然后,我从一张表(平均大约 30k)中获取所有 hte 列表并运行我的 ListingMigrator 方法。
Where do I put the chunk in this example? Would it replace the line:
在这个例子中我把块放在哪里?它会取代这一行:
$listings = $class::orderBy('MatrixModifiedDT','desc')->get();
I don't fully understand the closure in the eloquent docs. This is all they have to say about it and here's the code example from the Laravel site:
我不完全理解雄辩文档中的闭包。这就是他们要说的全部内容,这里是 Laravel 站点的代码示例:
User::chunk(200, function($users)
{
foreach ($users as $user)
{
//
}
});
回答by ceejayoz
The chunk
call should replace the get
call - it is designed to work on the pre-get() query builder object.
该chunk
调用应替换该get
调用 - 它旨在处理 pre-get() 查询构建器对象。
$listings = $class::orderBy('MatrixModifiedDT','desc');
$listings->chunk(200, function($listings) use($feed) {
$listings->each(function($listing) use ($feed) {
ListingMigrator::migrateListing($listing,$feed);
echo "Feed: $feed->board\r\n";
echo "SubcondoName: $listing->SubCondoName\r\n";
echo "Development: $listing->Development\r\n";
echo "\r\n";
});
});