异步运行 PHP 任务

Run PHP Task Asynchronously

提问by davr

I work on a somewhat large web application, and the backend is mostly in PHP. There are several places in the code where I need to complete some task, but I don't want to make the user wait for the result. For example, when creating a new account, I need to send them a welcome email. But when they hit the 'Finish Registration' button, I don't want to make them wait until the email is actually sent, I just want to start the process, and return a message to the user right away.

我在一个有点大的 web 应用程序上工作，后端主要是在 PHP 中。代码中有几个地方我需要完成一些任务，但我不想让用户等待结果。例如，在创建新帐户时，我需要向他们发送欢迎电子邮件。但是当他们点击“完成注册”按钮时，我不想让他们等到电子邮件实际发送时，我只想开始这个过程，并立即向用户返回一条消息。

Up until now, in some places I've been using what feels like a hack with exec(). Basically doing things like:

直到现在，在某些地方我一直在使用 exec() 感觉像是一个 hack 的东西。基本上做这样的事情：

exec("doTask.php $arg1 $arg2 $arg3 >/dev/null 2>&1 &");

Which appears to work, but I'm wondering if there's a better way. I'm considering writing a system which queues up tasks in a MySQL table, and a separate long-running PHP script that queries that table once a second, and executes any new tasks it finds. This would also have the advantage of letting me split the tasks among several worker machines in the future if I needed to.

这似乎有效，但我想知道是否有更好的方法。我正在考虑编写一个在 MySQL 表中排队任务的系统，以及一个单独的长时间运行的 PHP 脚本，该脚本每秒查询一次该表，并执行它找到的任何新任务。如果我需要，这也可以让我将来在几台工作机器之间分配任务。

Am I re-inventing the wheel? Is there a better solution than the exec() hack or the MySQL queue?

我是在重新发明轮子吗？有比 exec() hack 或 MySQL 队列更好的解决方案吗？

采纳答案by Paul Dixon

I've used the queuing approach, and it works well as you can defer that processing until your server load is idle, letting you manage your load quite effectively if you can partition off "tasks which aren't urgent" easily.

我使用了排队方法，它工作得很好，因为您可以推迟该处理直到您的服务器负载空闲，如果您可以轻松地划分“不紧急的任务”，则可以让您非常有效地管理您的负载。

Rolling your own isn't too tricky, here's a few other options to check out:

自己滚动并不太棘手，这里有一些其他选项可供查看：

• GearMan- this answer was written in 2009, and since then GearMan looks a popular option, see comments below.
• ActiveMQif you want a full blown open source message queue.
• ZeroMQ- this is a pretty cool socket library which makes it easy to write distributed code without having to worry too much about the socket programming itself. You could use it for message queuing on a single host - you would simply have your webapp push something to a queue that a continuously running console app would consume at the next suitable opportunity
• beanstalkd- only found this one while writing this answer, but looks interesting
• dropris a PHP based message queue project, but hasn't been actively maintained since Sep 2010
• php-enqueueis a recently (2017) maintained wrapper around a variety of queue systems
• Finally, a blog post about using memcached for message queuing

• GearMan- 这个答案写于 2009 年，从那时起，GearMan 看起来很受欢迎，请参阅下面的评论。
• 如果您想要一个完整的开源消息队列，请使用ActiveMQ。
• ZeroMQ- 这是一个非常酷的套接字库，它可以轻松编写分布式代码，而不必过多担心套接字编程本身。您可以将它用于单个主机上的消息队列 - 您只需让您的 web 应用程序将某些内容推送到一个队列中，连续运行的控制台应用程序将在下一个合适的机会使用该队列
• beanstalkd- 在写这个答案时才发现这个，但看起来很有趣
• dropr是一个基于 PHP 的消息队列项目，但自 2010 年 9 月以来一直没有积极维护
• php-enqueue是最近（2017 年）维护的围绕各种队列系统的包装器
• 最后，一篇关于使用memcached进行消息队列的博文

Another, perhaps simpler, approach is to use ignore_user_abort- once you've sent the page to the user, you can do your final processing without fear of premature termination, though this does have the effect of appearing to prolong the page load from the user perspective.

另一种可能更简单的方法是使用ignore_user_abort- 一旦您将页面发送给用户，您就可以进行最终处理而不必担心过早终止，尽管这确实会延长用户的页面加载时间看法。

回答by Markus

When you just want to execute one or several HTTP requests without having to wait for the response, there is a simple PHP solution, as well.

当您只想执行一个或多个 HTTP 请求而不必等待响应时，还有一个简单的 PHP 解决方案。

In the calling script:

在调用脚本中：

$socketcon = fsockopen($host, 80, $errno, $errstr, 10);
if($socketcon) {   
   $socketdata = "GET $remote_house/script.php?parameters=... HTTP 1.1\r\nHost: $host\r\nConnection: Close\r\n\r\n";      
   fwrite($socketcon, $socketdata); 
   fclose($socketcon);
}
// repeat this with different parameters as often as you like

On the called script.php, you can invoke these PHP functions in the first lines:

在被调用的 script.php 上，您可以在第一行调用这些 PHP 函数：

ignore_user_abort(true);
set_time_limit(0);

This causes the script to continue running without time limit when the HTTP connection is closed.

这会导致脚本在 HTTP 连接关闭时继续运行而不受时间限制。

回答by rojoca

Another way to fork processes is via curl. You can set up your internal tasks as a webservice. For example:

分叉进程的另一种方法是通过 curl。您可以将内部任务设置为网络服务。例如：

• http://domain/tasks/t1
• http://domain/tasks/t2

• http://域/任务/t1
• http://域/任务/t2

Then in your user accessed scripts make calls to the service:

然后在您的用户访问脚本中调用该服务：

$service->addTask('t1', $data); // post data to URL via curl

Your service can keep track of the queue of tasks with mysql or whatever you like the point is: it's all wrapped up within the service and your script is just consuming URLs. This frees you up to move the service to another machine/server if necessary (ie easily scalable).

您的服务可以使用 mysql 或任何您喜欢的方式跟踪任务队列：这一切都包含在服务中，而您的脚本只是使用 URL。这使您可以在必要时将服务移动到另一台机器/服务器（即易于扩展）。

Adding http authorization or a custom authorization scheme (like Amazon's web services) lets you open up your tasks to be consumed by other people/services (if you want) and you could take it further and add a monitoring service on top to keep track of queue and task status.

添加 http 授权或自定义授权方案（如 Amazon 的 Web 服务）可让您开放您的任务以供其他人/服务使用（如果您愿意），您可以更进一步并在顶部添加监控服务以跟踪队列和任务状态。

• http://domain/queue?task=t1
• http://domain/queue?task=t2
• http://domain/queue/t1/100931

• http://domain/queue?task=t1
• http://domain/queue?task=t2
• http://domain/queue/t1/100931

It does take a bit of set-up work but there are a lot of benefits.

这确实需要一些设置工作，但有很多好处。

回答by Alister Bulman

I've used Beanstalkdfor one project, and planned to again. I've found it to be an excellent way to run asynchronous processes.

我已经将Beanstalkd用于一个项目，并计划再次使用。我发现它是运行异步进程的绝佳方式。

A couple of things I've done with it are:

我用它做的几件事是：

• Image resizing - and with a lightly loaded queue passing off to a CLI-based PHP script, resizing large (2mb+) images worked just fine, but trying to resize the same images within a mod_php instance was regularly running into memory-space issues (I limited the PHP process to 32MB, and the resizing took more than that)
• near-future checks - beanstalkd has delays available to it (make this job available to run only after X seconds) - so I can fire off 5 or 10 checks for an event, a little later in time

• 图像大小调整 - 并且将轻量加载队列传递给基于 CLI 的 PHP 脚本，调整大（2mb+）图像大小工作得很好，但尝试在 mod_php 实例中调整相同图像大小经常遇到内存空间问题（我将 PHP 进程限制为 32MB，而调整大小需要更多）
• 近期检查 - beanstalkd 有可用的延迟（使此作业仅在 X 秒后才能运行） - 所以我可以在稍晚一点的时间为一个事件触发 5 或 10 次检查

I wrote a Zend-Framework based system to decode a 'nice' url, so for example, to resize an image it would call QueueTask('/image/resize/filename/example.jpg'). The URL was first decoded to an array(module,controller,action,parameters), and then converted to JSON for injection to the queue itself.

我编写了一个基于 Zend-Framework 的系统来解码一个“不错”的 url，例如，要调整它会调用的图像大小QueueTask('/image/resize/filename/example.jpg')。URL首先被解码为一个数组（模块，控制器，动作，参数），然后转换为JSON注入到队列本身。

A long running cli script then picked up the job from the queue, ran it (via Zend_Router_Simple), and if required, put information into memcached for the website PHP to pick up as required when it was done.

一个长时间运行的 cli 脚本然后从队列中提取作业，运行它（通过 Zend_Router_Simple），如果需要，将信息放入 memcached 以便网站 PHP 在完成时根据需要提取。

One wrinkle I did also put in was that the cli-script only ran for 50 loops before restarting, but if it did want to restart as planned, it would do so immediately (being run via a bash-script). If there was a problem and I did exit(0)(the default value for exit;or die();) it would first pause for a couple of seconds.

我也遇到的一个问题是 cli 脚本在重新启动之前只运行了 50 个循环，但如果它确实想按计划重新启动，它会立即这样做（通过 bash 脚本运行）。如果出现问题并且我做了exit(0)（exit;或的默认值die();），它将首先暂停几秒钟。

回答by Denys Gorobchenko

If it just a question of providing expensive tasks, in case of php-fpm is supported, why not to use fastcgi_finish_request()function?

如果只是提供昂贵任务的问题，在支持php-fpm的情况下，为什么不使用fastcgi_finish_request()函数？

This function flushes all response data to the client and finishes the request. This allows for time consuming tasks to be performed without leaving the connection to the client open.

此函数将所有响应数据刷新到客户端并完成请求。这允许在不保持与客户端的连接打开的情况下执行耗时的任务。

You don't really use asynchronicity in this way:

您并没有真正以这种方式使用异步性：

1. Make all your main code first.
2. Execute fastcgi_finish_request().
3. Make all heavy stuff.

1. 首先制作所有主要代码。
2. 执行fastcgi_finish_request()。
3. 制作所有重物。

Once again php-fpm is needed.

再次需要 php-fpm。

回答by Andrew Moore

Here is a simple class I coded for my web application. It allows for forking PHP scripts and other scripts. Works on UNIX and Windows.

这是我为我的 Web 应用程序编写的一个简单类。它允许分叉 PHP 脚本和其他脚本。适用于 UNIX 和 Windows。

class BackgroundProcess {
    static function open($exec, $cwd = null) {
        if (!is_string($cwd)) {
            $cwd = @getcwd();
        }

        @chdir($cwd);

        if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
            $WshShell = new COM("WScript.Shell");
            $WshShell->CurrentDirectory = str_replace('/', '\', $cwd);
            $WshShell->Run($exec, 0, false);
        } else {
            exec($exec . " > /dev/null 2>&1 &");
        }
    }

    static function fork($phpScript, $phpExec = null) {
        $cwd = dirname($phpScript);

        @putenv("PHP_FORCECLI=true");

        if (!is_string($phpExec) || !file_exists($phpExec)) {
            if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
                $phpExec = str_replace('/', '\', dirname(ini_get('extension_dir'))) . '\php.exe';

                if (@file_exists($phpExec)) {
                    BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
                }
            } else {
                $phpExec = exec("which php-cli");

                if ($phpExec[0] != '/') {
                    $phpExec = exec("which php");
                }

                if ($phpExec[0] == '/') {
                    BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
                }
            }
        } else {
            if (strtoupper(substr(PHP_OS, 0, 3)) == 'WIN') {
                $phpExec = str_replace('/', '\', $phpExec);
            }

            BackgroundProcess::open(escapeshellarg($phpExec) . " " . escapeshellarg($phpScript), $cwd);
        }
    }
}

回答by Darryl Hein

This is the same method I have been using for a couple of years now and I haven't seen or found anything better. As people have said, PHP is single threaded, so there isn't much else you can do.

这与我已经使用了几年的方法相同，但我没有看到或发现更好的方法。正如人们所说，PHP 是单线程的，因此您无能为力。

I have actually added one extra level to this and that's getting and storing the process id. This allows me to redirect to another page and have the user sit on that page, using AJAX to check if the process is complete (process id no longer exists). This is useful for cases where the length of the script would cause the browser to timeout, but the user needs to wait for that script to complete before the next step. (In my case it was processing large ZIP files with CSV like files that add up to 30 000 records to the database after which the user needs to confirm some information.)

我实际上为此添加了一个额外的级别，即获取和存储进程 ID。这允许我重定向到另一个页面并让用户坐在该页面上，使用 AJAX 检查进程是否完成（进程 ID 不再存在）。这对于脚本的长度会导致浏览器超时的情况很有用，但用户需要在下一步之前等待该脚本完成。（在我的情况下，它正在处理带有 CSV 的大型 ZIP 文件，例如将多达 30 000 条记录添加到数据库中的文件，之后用户需要确认一些信息。）

I have also used a similar process for report generation. I'm not sure I'd use "background processing" for something such as an email, unless there is a real problem with a slow SMTP. Instead I might use a table as a queue and then have a process that runs every minute to send the emails within the queue. You would need to be warry of sending emails twice or other similar problems. I would consider a similar queueing process for other tasks as well.

我也使用了类似的过程来生成报告。我不确定我是否会对电子邮件之类的东西使用“后台处理”，除非慢速 SMTP 存在真正的问题。相反，我可能会使用一个表作为队列，然后有一个每分钟运行一次的进程来发送队列中的电子邮件。您需要警惕两次发送电子邮件或其他类似问题。我也会为其他任务考虑类似的排队过程。

回答by Omar S.

PHP HASmultithreading, its just not enabled by default, there is an extension called pthreadswhich does exactly that. You'll need php compiled with ZTS though. (Thread Safe) Links:

PHP有多线程，只是默认情况下没有启用，有一个名为pthreads的扩展可以做到这一点。不过，您需要使用 ZTS 编译 php。（线程安全）链接：

Examples

例子

Another tutorial

另一个教程

pthreads PECL Extension

pthreads PECL 扩展

回答by Kjeld

It's a great idea to use cURL as suggested by rojoca.

按照 rojoca 的建议使用 cURL 是个好主意。

Here is an example. You can monitor text.txt while the script is running in background:

这是一个例子。您可以在脚本在后台运行时监视 text.txt：

<?php

function doCurl($begin)
{
    echo "Do curl<br />\n";
    $url = 'http://'.$_SERVER['SERVER_NAME'].$_SERVER['REQUEST_URI'];
    $url = preg_replace('/\?.*/', '', $url);
    $url .= '?begin='.$begin;
    echo 'URL: '.$url.'<br>';
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $result = curl_exec($ch);
    echo 'Result: '.$result.'<br>';
    curl_close($ch);
}


if (empty($_GET['begin'])) {
    doCurl(1);
}
else {
    while (ob_get_level())
        ob_end_clean();
    header('Connection: close');
    ignore_user_abort();
    ob_start();
    echo 'Connection Closed';
    $size = ob_get_length();
    header("Content-Length: $size");
    ob_end_flush();
    flush();

    $begin = $_GET['begin'];
    $fp = fopen("text.txt", "w");
    fprintf($fp, "begin: %d\n", $begin);
    for ($i = 0; $i < 15; $i++) {
        sleep(1);
        fprintf($fp, "i: %d\n", $i);
    }
    fclose($fp);
    if ($begin < 10)
        doCurl($begin + 1);
}

?>

回答by phpPhil

If you don't want the full blown ActiveMQ, I recommend to consider RabbitMQ. RabbitMQ is lightweight messaging that uses the AMQP standard.

如果您不想要完整的 ActiveMQ，我建议您考虑RabbitMQ。RabbitMQ 是使用AMQP 标准的轻量级消息传递。

I recommend to also look into php-amqplib- a popular AMQP client library to access AMQP based message brokers.

我建议还查看php-amqplib- 一个流行的 AMQP 客户端库，用于访问基于 AMQP 的消息代理。