javascript 有没有办法让 cURL 等到页面的动态更新完成？

Is there a way to let cURL wait until the page's dynamic updates are done?

提问by

I'm fetching pages with cURL in PHP. Everything works fine, but I'm fetching some parts of the page that are calculated with JavaScript a fraction after the page is loaded. cURL already send the page's source back to my PHP script before the JavaScript calculations are done, thus resulting in wrong end-results. The calculations on the site are fetched by AJAX, so I can't reproduce that calculation in an easy way. Also I have no access to the target-page's code, so I can't tweak that target-page to fit my (cURL) fetching needs.

我在 PHP 中使用 cURL 获取页面。一切正常，但我正在获取页面加载后一小部分用 JavaScript 计算的页面部分。在 JavaScript 计算完成之前，cURL 已经将页面的源代码发送回我的 PHP 脚本，从而导致错误的最终结果。网站上的计算是由 AJAX 获取的，因此我无法以简单的方式重现该计算。此外，我无法访问目标页面的代码，因此我无法调整该目标页面以满足我的（cURL）获取需求。

Is there any way I can tell cURL to wait until all dynamic traffic is finished? It might be tricky, due to some JavaScripts that are keep sending data back to another domain that might result in long hangs. But at least I can test then if I at least get the correct results back.

有什么办法可以让 cURL 等到所有动态流量都完成？这可能很棘手，因为一些 JavaScript 会不断将数据发送回另一个域，这可能会导致长时间挂起。但至少我可以测试如果我至少得到正确的结果。

My Developer toolbar in Safari indicates the page is done in about 1.57s. Maybe I can tell cURL statically to wait for 2 seconds too?

我在 Safari 中的开发者工具栏指示页面在大约 1.57 秒内完成。也许我也可以静态地告诉 cURL 等待 2 秒？

I wonder what the possibilities are :)

我想知道有什么可能性:)

采纳答案by Jan Han?i?

cURL does not execute any JavaScript or download any files referenced in the document. So cURL is not the solution for your problem.

cURL 不执行任何 JavaScript 或下载文档中引用的任何文件。所以 cURL 不是您问题的解决方案。

You'll have to use a browser on the server side, tell it to load the page, wait for X seconds and then ask it to give you the HTML.

您必须在服务器端使用浏览器，告诉它加载页面，等待 X 秒，然后要求它为您提供 HTML。

Look at: http://phantomjs.org/(you'll need to use node.js, I'm not aware of any PHP solutions).

查看：http: //phantomjs.org/（您需要使用 node.js，我不知道任何 PHP 解决方案）。

回答by Peter Herdenborg

Not knowing a lot about the page you are retrieving or the calculations you want to include, but it could be an option to cURL straight to the URL serving those ajax requests. Use something like Firebug to inspect the Ajax calls being made on your target page and you can figure out the URL and any parameters passed. If you do need the full web page, maybe you can cURL both the web page and the Ajax URL and combine the two in your PHP code, but then it starts to get messy.

不太了解您正在检索的页面或您想要包含的计算，但它可能是一个选项，可以直接 cURL 到为这些 ajax 请求提供服务的 URL。使用 Firebug 之类的工具来检查在目标页面上进行的 Ajax 调用，您可以找出 URL 和传递的任何参数。如果您确实需要完整的网页，也许您可​​以同时对网页和 Ajax URL 进行 cURL 并将两者结合到您的 PHP 代码中，但随后它开始变得混乱。

回答by K. Igor

There is one quite tricky way to achieve it using php. If you' really like it to work for php you could potentially use Codeception setup in junction with Selenium and use Chrome browser webdriver in headless mode.

使用 php 有一种非常棘手的方法来实现它。如果你真的喜欢它为 php 工作，你可能会使用 Codeception 设置与 Selenium 结合并在无头模式下使用 Chrome 浏览器 webdriver。

Here are some general steps to have it working.

以下是一些使其正常工作的一般步骤。

1. You make sure you have codeception in your PHP project https://codeception.com

2. Download chrome webdriver: https://chromedriver.chromium.org/downloads

3. Download selenium: https://www.seleniumhq.org/download/

4. Configure it accordingly looking into documentation of codeception framework.

5. Write codeception test where you can use expression like $I->wait(5)for waiting 5 seconds or $I->waitForJs('js expression here')for waiting for js script to complete on the page.

6. Run written in previous step test using command php vendor/bin/codecept path/to/test

1. 你确保你的 PHP 项目中有 codeception https://codeception.com

2. 下载 chrome 网络驱动程序：https://chromedriver.chromium.org/downloads

3. 下载硒：https: //www.seleniumhq.org/download/

4. 相应地配置它，查看代码接收框架的文档。

5. 编写代码接收测试，您可以在其中使用表达式，例如$I->wait(5)等待 5 秒或$I->waitForJs('js expression here')等待 js 脚本在页面上完成。

6. 使用命令运行在上一步测试中编写的 php vendor/bin/codecept path/to/test