php 卷曲：传输关闭，剩余未读数据

Question

提问by

I'm having an issue with a large curl call.

我遇到了大卷曲调用的问题。

I get

我得到

nread <= 0, server closed connection, bailing
transfer closed with outstanding read data remaining

nread <= 0，服务器关闭连接，保释
传输关闭，剩余未完成的读取数据

and the content is partial delivered

并且内容是部分交付的

    GET /stats/?stats_breakdown=track__track&campaign=&search_criteria=2&period=0&date_month=11&date_day=03&date_year=2015&start_date_month=11&start_date_day=03&start_date_year=2015&end_date_month=12&end_date_day=31&end_date_year=2014 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13
Host: domain.com
Accept: */*
Cookie: sessionid=xxg4gglsm7o3b224wihqz8od19wl31h1; csrftoken=JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C
Cache-Control: no-cache
Connection: Keep-Alive
Keep-Alive: 600
Accept-Language: en-us
X-CSRFToken: SeN9bHryRK8FWLTLJIs5c6u9AZ47a8pR
Content-Type: application/x-www-form-urlencoded
Origin: https://domain.com
Referer: https://domain.com

* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Server: nginx/1.8.0
< Date: Wed, 04 Nov 2015 12:54:05 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Vary: Cookie, Accept-Language
< P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
< Content-Language: en
* Replaced cookie csrftoken="JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C" for domain domain.com, path /, expire 1478091245
< Set-Cookie: csrftoken=JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C; expires=Wed, 02-Nov-2016 12:54:05 GMT; Max-Age=31449600; Path=/; secure
<
* nread <= 0, server closed connection, bailing
* transfer closed with outstanding read data remaining
* Closing connection #0

This is the php configuration I use

这是我使用的php配置

function getHeaders()
{
    $headers = array();
    $headers[] = 'Cache-Control: no-cache';
    $headers[] = 'Connection: Keep-Alive';
    $headers[] = 'Keep-Alive: 600';
    $headers[] = 'Accept-Language: en-us';
    $headers[] = 'X-CSRFToken: SeN9bHryRK8FWLTLJIs5c6u9AZ47a8pR';

    $headers[] = 'Content-Type: application/x-www-form-urlencoded';
    $headers[] = 'Origin: https://domain.com';
    $headers[] = 'Referer: https://domain.com';

    return $headers;

}

curl_setopt($connection, CURLOPT_URL, $url);
    curl_setopt($connection, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($connection, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($connection, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
    curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($connection, CURLOPT_SSL_VERIFYHOST, false);
    curl_setopt($connection, CURLOPT_COOKIEFILE, 'cookie.txt');
    curl_setopt($connection, CURLOPT_COOKIEJAR, 'cookie.txt');
    curl_setopt($connection, CURLOPT_CONNECTTIMEOUT ,550000000);
    curl_setopt($connection, CURLOPT_TIMEOUT, 5500000000); //timeout in seconds
    curl_setopt($connection, CURLOPT_HTTPHEADER, getHeaders());
    curl_setopt($connection, CURLOPT_VERBOSE, 1);

Answer 1

采纳答案by

Okay, after some searching and IRC chat's I found solution, but not 100% sure what the cause is. Looks like the keep-alives weren't send enough to keep the connection going on. Will post the solution here, hopefully I can help someone out.

好的，经过一番搜索和 IRC 聊天后，我找到了解决方案，但不能 100% 确定原因是什么。看起来keep-alives发送的不够多，无法保持连接继续。将在这里发布解决方案，希望我可以帮助某人。

What helped for me is adding

对我有帮助的是添加

--keepalive-time 2

An explanation of the curl option

curl 选项的解释

--keepalive-time <seconds>

This option sets the time a connection needs to remain idle before sending keepalive probes and the time between individual keepalive probes. It is currently effective on operating systems offering the TCP_KEEPIDLE and TCP_KEEPINTVL socket options (meaning Linux, recent AIX, HP-UX and more). This option has no effect if --no-keepalive is used. (Added in 7.18.0)
If this option is used several times, the last one will be used. If unspecified, the option defaults to 60 seconds.

此选项设置连接在发送 keepalive 探测之前需要保持空闲的时间以及各个 keepalive 探测之间的时间。它目前在提供 TCP_KEEPIDLE 和 TCP_KEEPINTVL 套接字选项的操作系统上有效（意味着 Linux、最近的 AIX、HP-UX 等）。如果使用 --no-keepalive，则此选项无效。（在 7.18.0 中添加）
如果多次使用此选项，将使用最后一个。如果未指定，该选项默认为 60 秒。

Looks like the default was too high to keep my connection open.

看起来默认值太高而无法保持我的连接打开。

Here is the full command I used for my call

这是我用于通话的完整命令

curl URL -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8,et;q=0.6,nl;q=0.4' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Connection: keep-alive' --compressed -v --keepalive-time 2

And I'm running this version of curl on osx

我在 osx 上运行这个版本的 curl

curl 7.43.0 (x86_64-apple-darwin15.0) libcurl/7.43.0 SecureTransport zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets

If someone would like to have this option used in PHP curl, the --keepalive-time option is available since PHP 5.5. You can use it as following:

如果有人想在 PHP curl 中使用这个选项，从 PHP 5.5 开始就可以使用 --keepalive-time 选项。您可以按以下方式使用它：

curl_setopt($connection, CURLOPT_TCP_KEEPALIVE, 1);
curl_setopt($connection, CURLOPT_TCP_KEEPIDLE, 2);

Hope this helps someone struggling with the same issue!

希望这可以帮助遇到同样问题的人！

Answer 2

回答by Hemanth

Encountered similar issue, my server is behind nginx. Curl is able to receive a response if it connects to the server directly but if curl connects to the server via nginx, curl is throwing below error

遇到类似问题，我的服务器在 nginx 后面。如果 curl 直接连接到服务器，则它能够收到响应，但如果 curl 通过 nginx 连接到服务器，则 curl 将抛出以下错误

Session
* transfer closed with outstanding read data remaining
* Closing connection 0 curl: (18) transfer closed with outstanding read data remaining

会话
* 传输关闭，剩余未完成的读取数据
* 关闭连接 0 curl: (18) 传输关闭，剩余未完成的读取数据

When I connect to same nginx URL using a browser, Response is showing up fine. It is very weird, When I tried to connect to the same nginx URL using curl, it was throwing above error.

当我使用浏览器连接到同一个 nginx URL 时，响应显示正常。很奇怪，当我尝试使用 curl 连接到同一个 nginx URL 时，它抛出了上述错误。

After comparing headers sent by the browser and curl. I found that the browser is able to receive the response because of the below header which curl is not sending:

比较浏览器和 curl 发送的标头后。我发现浏览器能够接收响应，因为 curl 没有发送以下标头：

'Accept-Encoding: gzip'

Sending above header using curl is working fine. So, what the above header is doing is compressing response to gzip thus reducing the response size.

使用 curl 发送上面的标题工作正常。因此，上述标头所做的是将响应压缩到 gzip，从而减小响应大小。

After some more digging, found that nginx is not able to send any payload > 80kb. After wasting a lot of time, found that issue is with nginx buffering and nginx worked like a charm after adding below proxy_buffering property in nginx.conf:

经过更多的挖掘，发现nginx无法发送任何大于80kb的有效负载。浪费了很多时间后，发现问题出在 nginx 缓冲上，并且在 nginx.conf 中添加以下 proxy_buffering 属性后，nginx 工作起来就像一个魅力：

 location / {
 proxy_buffering off;
 }

The accepted answer didn't resolve my issue. Writing this answer, so that anyone need not waste their time if they are facing the same issue as I did.

接受的答案没有解决我的问题。写下这个答案，以便任何人在遇到和我一样的问题时都不必浪费时间。

Answer 3

回答by Daniel Stenberg

libcurl only tells you that the connection was cut by the server in an unclean way where it didn't deliver data it had promised to do. It looks like the chunked-encoding didn't signal the end of the transfer.

libcurl 只告诉您连接被服务器以不干净的方式切断，它没有提供它承诺的数据。看起来分块编码没有表示传输结束。

Browsers are notorious for being extremely liberal in what they receive so they overlook and are fine with all sorts of protocol violations to a much larger extent than libcurl is.

众所周知，浏览器在接收到的内容方面非常自由，因此它们会忽略并在比 libcurl 更大的范围内处理各种违反协议的行为。

Answer 4

回答by FeiXia

Encountered similar issue, my server is also behind nginx. There's no error in web server's (flask) log, but some error messsages are found in nginx log.

遇到类似问题，我的服务器也在nginx后面。web服务器（flask）日志中没有错误，但是在nginx日志中发现了一些错误消息。

[crit] 31054#31054: *269464 open() "/var/cache/nginx/proxy_temp/3/45/0000000453" failed (13: Permission denied) while reading upstream

[crit] 31054#31054: *269464 open() "/var/cache/nginx/proxy_temp/3/45/0000000453" 在读取上游时失败（13：权限被拒绝）

I fix this issue by correcting the permission of directory '/var/cache/nginx'

我通过更正目录“/var/cache/nginx”的权限来解决这个问题

php 卷曲：传输关闭，剩余未读数据

提问by

采纳答案by

回答by Hemanth

回答by Daniel Stenberg

回答by FeiXia

相关推荐

最近更新

标签

php 卷曲：传输关闭，剩余未读数据

提问by

采纳答案by

回答by Hemanth

回答by Daniel Stenberg

回答by FeiXia

相关推荐

php 如何将带有日期和时间 AM/PM 的字符串转换为 24 小时 mysql 时间戳格式

php 如何从php中的数组中找到平均值？

从 PHP 执行 wkhtmltopdf

php 警告：mysqli_connect()：MySQL 服务器已经消失

相关推荐

最近更新

标签