使用 Supervisor 运行多个 Laravel 队列工作器

Running multiple Laravel queue workers using Supervisor

提问by Ryk Waters

I using Laravel queues using a database driver and supervisor to keep a queue worker running all the time:

我使用 Laravel 队列使用数据库驱动程序和主管来保持队列工作者一直运行：

[program:laravel_queue]
command=php artisan queue:listen --timeout=1800 --tries=5
directory=/var/app/current
stdout_logfile=/var/app/support/logs/laravel-queue.log
logfile_maxbytes=0
logfile_backups=0
redirect_stderr=true
autostart=true
autorestart=true
startretries=86400
EOB

Some of the queue tasks can take around 10 minutes to complete.

某些队列任务可能需要大约 10 分钟才能完成。

I have 2 parts to the question:

我有两个部分的问题：

1) How can i edit the above script to run multiple (e.g. 3) queue workers on the same queue.

1) 如何编辑上述脚本以在同一队列上运行多个（例如 3 个）队列工作器。

2) Is there a way of scaling the number of queue workers running based on the number of jobs waiting to be processed?

2) 有没有一种方法可以根据等待处理的作业数量来扩展正在运行的队列工作器的数量？

The reason for question 2 is that we have batches of busy times and then lots of quiet times, so i don't really want to be wasting resources with 3 listeners running the whole time.

问题 2 的原因是我们有一批繁忙的时间，然后有很多安静的时间，所以我真的不想浪费资源，同时有 3 个听众一直在运行。

回答by Nestor Mata Cuthbert

In supervisor you specify the amount of processes with the parameter numprocs, so you can add a line to your script that says:

在 supervisor 中，您使用参数指定进程数量numprocs，因此您可以在脚本中添加一行，内容为：

numprocs=5

numprocs=5

Now, you can do some clever things, for example if only some of the processes that runs on the queues takes too long, you can create a different set of queues processes to work those and other set for the light processes. To achieve that you can create a supervisor configuration with one queue name like --queue=longprocessand another one with --queue=lightprocessand in your program you dispatch the job in the corresponding queue, that way, long processes won't delay short processes.

现在，您可以做一些聪明的事情，例如，如果只有一些在队列上运行的进程花费的时间太长，您可以创建一组不同的队列进程来为轻进程工作。为了实现这一目标，你可以创建一个主管的配置与像一个队列名称--queue=longprocess和另一个--queue=lightprocess和你的程序，你在相应的队列调度工作，这样一来，一个漫长的过程不会耽误很短的过程。

You can also specify queue prioritieswithin one supervisor configuration file such as --queue=lightprocess,longprocess. That way, your worker(s) will first look for lightprocessbefore running longprocess.

您还可以在一个主管配置文件中指定队列优先级，例如--queue=lightprocess,longprocess. 这样，您的工作人员将lightprocess在运行之前首先查找longprocess。

To answer your second question, no, in terms of supervisor all processes are running, it doesn't knows if the queue is busy or idle, therefore it can't kill processes and create them based on their use, so no, you can't have a dynamic configuration of creating more processes only when the ones you have are busy.

要回答你的第二个问题，不，就主管而言，所有进程都在运行，它不知道队列是忙还是闲，因此它不能杀死进程并根据它们的使用情况创建它们，所以不，你可以没有动态配置，只有在您拥有的进程忙碌时才创建更多进程。

A note, if you assign more than 1 numprocs, you must add the number of process to the name. According to supervisor configuration docs:

请注意，如果您分配的 numprocs 超过 1 个，则必须将进程数添加到名称中。根据主管配置文档：

Supervisor will start as many instances of this program as named by numprocs. Note that if numprocs > 1, the process_name expression must include %(process_num)s (or any other valid Python string expression that includes process_num) within it.

Supervisor 将启动与 numprocs 命名的该程序的多个实例。请注意，如果 numprocs > 1，则 process_name 表达式中必须包含 %(process_num)s（或任何其他包含 process_num 的有效 Python 字符串表达式）。

Supervisor configuration docs: http://supervisord.org/configuration.html

主管配置文档：http: //supervisord.org/configuration.html

回答by kanhaiya charka

To run the multiple jobs automatically using supervisor add numprocs = 5, this means supervisor run 5 processes at a time and monitor all of them. To know more about this visit - https://cloudtoolz.io/queue-laravel-in-database/

要使用 supervisor add numprocs = 5 自动运行多个作业，这意味着 supervisor 一次运行 5 个进程并监视所有进程。要了解有关此访问的更多信息 - https://cloudtoolz.io/queue-laravel-in-database/

回答by Curvian Vynes

From Laravel docsyou can use numprocs=3to spawn 3 processes.

从Laravel 文档中，您可以numprocs=3用来生成 3 个进程。

And also specify a queue: command=php artisan queue:listen --queue=myqueue --tries=5

并指定一个队列： command=php artisan queue:listen --queue=myqueue --tries=5

回答by Satish Shinde

In supervisor config file. Add following code

在主管配置文件中。添加以下代码

process_name=%(program_name)s_%(process_num)02d
numprocs=8

This will create the 8 different processes of your program. You just need to change the value of numprocs=8if you want.

这将创建程序的 8 个不同进程。如果需要，您只需要更改 的值numprocs=8。

You can check how many processes are running with sudo supervisorctl status

您可以检查有多少进程正在运行 sudo supervisorctl status