Laravel Artisan 队列 - 高 CPU 使用率

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/16897997/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-14 07:52:30  来源:igfitidea点击:

Laravel Artisan Queues - high cpu usage

laravellaravel-4supervisordbeanstalkd

提问by bsparacino

I have set up queues in Laravel for my processing scripts. I am using beanstalkd and supervisord. There are 6 different tubes for different types of processing.

我已经在 Laravel 中为我的处理脚本设置了队列。我正在使用 beanstalkd 和 supervisord。有 6 种不同的管子用于不同类型的加工。

The issue is that for each tube, artisan is constantly spawning workers every second. The worker code seems to sleep for 1 second and then the worker thread uses 7-15% cpu, multiply this by 6 tubes... and I would like to have multiple workers per tube.. my cpu is being eaten up.

问题是,对于每个管子,工匠每秒都在不断地产生工人。工作代码似乎睡了 1 秒,然后工作线程使用 7-15% 的 cpu,乘以 6 个管……我希望每个管有多个工人……我的 cpu 被吃光了。

I tried changing the 1 second sleep to 10 seconds. This helps but there is still a huge cpu spike every 10 seconds when the workers wake back up. I am not even processing anything at this time because the queues are completely empty, it is simply the workers looking for something to do.

我尝试将 1 秒睡眠更改为 10 秒。这有帮助,但是当工作人员醒来时,每 10 秒仍然会有一个巨大的 CPU 峰值。我现在什至没有处理任何事情,因为队列完全是空的,这只是工作人员在寻找可以做的事情。

I also tested to see the cpu usage of laravel when I refreshed the page in a brower and that was hovering around 10%.. I am on a low end rackspace instance right now so that could explain it but still... it seems like the workers spin up a laravel instance every time they wake up.

我还测试了当我在浏览器中刷新页面时 laravel 的 CPU 使用情况,并且在 10% 左右徘徊。工人每次醒来时都会启动一个 Laravel 实例。

Is there no way to solve this? Do I just have to put a lot of money into a more expensive server just to be able to listen to see if a job is ready?

有没有办法解决这个问题?我是否只需要在更昂贵的服务器上投入大量资金才能听取工作是否准备就绪?

EDIT:

编辑:

Found a solution... it was to NOT use the artisan queue:listener or queue:work I looked into the queue code and there doesn't seem to be a way around this issue, it requires laravel to load every time a worker checks for more work to do.

找到了一个解决方案......它是不使用工匠队列:侦听器或队列:工作我查看了队列代码,似乎没有解决此问题的方法,每次工作人员检查时都需要加载laravel更多工作要做。

Instead I wrote my own listener using pheanstalk. I am still using laravel to push things into the queue, then my custom listener is parsing the queue data and then triggering an artisan command to run.

相反,我使用 pheanstalk 编写了自己的监听器。我仍在使用 laravel 将内容推送到队列中,然后我的自定义侦听器正在解析队列数据,然后触发运行 artisan 命令。

Now my cpu usage for my listeners is under %0, the only time my cpu shoots up now is when it actually finds work to do and then triggers the command, I am fine with that.

现在我的听众的 cpu 使用率低于 %0,我的 cpu 现在唯一一次突然上升是当它真正找到工作然后触发命令时,我很好。

回答by MohitMamoria

The problem of high CPU is caused because the worker loads the complete framework everytime it checks for a job in the queue. In laravel 4.2, you can use php artisan queue:work --daemon. This will load the framework once and the checking/processing of jobs happen inside a whileloop, which lets CPU breathe easy. You can find more about daemon worker in the official documentation: http://laravel.com/docs/queues#daemon-queue-worker.

CPU 高的问题是因为worker 每次检查队列中的作业时都会加载完整的框架。在 Laravel 4.2 中,您可以使用php artisan queue:work --daemon. 这将加载一次框架,并且作业的检查/处理在while循环内发生,这让 CPU 喘不过气来。您可以在官方文档中找到有关 daemon worker 的更多信息:http: //laravel.com/docs/queues#daemon-queue-worker

However, this benefit comes with a drawback - you need special care when deploying the code and you have to take care of the database connections. Usually, long running database connections are disconnected.

但是,这种好处也有一个缺点——在部署代码时需要特别小心,并且必须注意数据库连接。通常,长时间运行的数据库连接会断开。

回答by karelv

I had the same issue.

我遇到过同样的问题。

But I found another solution. I used the artisan worker as is, but I modified the 'watch' time. By default(from laravel) this time is hardcoded to zero, I've changed this value to 600 (seconds). See the file: 'vendor/laravel/framework/src/Illuminate/Queue/BeanstalkdQueue.php' and in function 'public function pop($queue = null)'

但我找到了另一个解决方案。我按原样使用了工匠,但我修改了“观看”时间。默认情况下(来自 laravel)这次硬编码为零,我已将此值更改为 600(秒)。查看文件:'vendor/laravel/framework/src/Illuminate/Queue/BeanstalkdQueue.php'和函数'public function pop($queue = null)'

So now the work is also listening to the queue for 10 minutes. When it does not have a job, it exits, and supervisor is restarting it. When it receives a job, it executes it after that it exists, and supervisor is restarting it.

所以现在工作也在监听队列10分钟。当它没有作业时,它退出,主管正在重新启动它。当它收到一个作业时,它会在它存在之后执行它,并且主管正在重新启动它。

==> No polling anymore!

==> 没有投票了!

notes:

笔记:

  • it does not work for iron.io queue's or others.
  • it might not work when you want that 1 worker accept jobs from more than 1 queue.
  • 它不适用于 Iron.io 队列或其他队列。
  • 当您希望 1 个工人接受来自 1 个以上队列的作业时,它可能不起作用。

回答by younes0

According to this commit, you can now set a new option to queue:listen "--sleep={int}"which will let you fine tune how much time to wait before polling for new jobs.
Also, default has been set to 3 instead of 1.

根据此提交,您现在可以设置一个新选项queue:listen "--sleep={int}",让您可以微调轮询新作业之前等待的时间。
此外,默认值已设置为 3 而不是 1。