Java HttpClient 在访问大块资源时抛出 TruncatedChunkException

HttpClient throws TruncatedChunkException accessing large chunked resource

提问by Phasmal

[using httpcore 4.1.4, httpclient 4.2.5, Oracle JDK 1.7.0_25]

[使用 httpcore 4.1.4、httpclient 4.2.5、Oracle JDK 1.7.0_25]

I'm trying to 'proxy' a connection to a third party web service on behalf of a webapp's javascript (AJAX) code and it seems to fail on large chunkedresponses, erroring part way through a chunk by sending multiple RSTs and throwing a org.apache.http.TruncatedChunkException.

我正在尝试代表 webapp 的 javascript (AJAX) 代码“代理”到第三方 web 服务的连接，但它似乎在大型chunked响应中失败，通过发送多个 RST 并抛出一个org.apache.http.TruncatedChunkException.

So I'm wondering:

所以我想知道：

1. why is http client trying to drop the connection?
2. is it doing something sensible? (ie. is the server possibly at fault) or is there something buggy going on here?

1. 为什么 http 客户端试图断开连接？
2. 它在做一些明智的事情吗？（即服务器可能有问题）还是这里发生了什么问题？

My basic approach is to copy everything from a servlet's request object to an apache components httpclient request and execute. More specfically, I:

我的基本方法是将所有内容从 servlet 的请求对象复制到 apache 组件 httpclient 请求并执行。更具体地说，我：

1. create an apache commons httpclient DefaultHttpClient object,
2. copy all request headers across to a new request object,
3. set (/override) the hostheader on the new request with the host/port I'm proxying to,
4. copy all HTTP parameters to the new request,
5. copy any entity body across to the new request,
6. execute the request,
7. copy response headers to my servlet's response headers, and
8. copy any entity body as a stream across to the servlet's output stream.

1. 创建一个 apache commons httpclient DefaultHttpClient 对象，
2. 将所有请求标头复制到新的请求对象中，
3. host使用我代理的主机/端口设置（/覆盖）新请求的标头，
4. 将所有 HTTP 参数复制到新请求中，
5. 将任何实体主体复制到新请求中，
6. 执行请求，
7. 将响应头复制到我的 servlet 的响应头中，并且
8. 将任何实体主体作为流复制到 servlet 的输出流。

The bit that is causing me issues is the last one. It seems to fail half way through a chunk and I get the following stacktrace:

引起我问题的一点是最后一个。它似乎在一个块中途失败，我得到以下堆栈跟踪：

org.apache.http.TruncatedChunkException: Truncated chunk ( expected size: 7752; actual size: 4077)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:186)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
at <mypackage>.<MyServlet>.service(<MyServlet>.java:XXX)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.doFilter(FilterDispatcher.java:63)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:724)

I've snooped it with Wireshark and get a process something like this:

我用 Wireshark 窥探过它，并得到了一个类似这样的过程：

source  dest    info
client  server  [SYN] seq=0
server  client  [SYN, ACK] seq=0 ack=1
client  server  [ACK] seq=1 ack=1
client  server  GET /url?param=value... HTTP/1.1
server  client  [ACK] seq=1 ack=221
server  client  [TCP segment of a reassembled PDU]
client  server  [ACK] seq=221 ack=4345
client  server  [FIN, ACK] seq=221 ack=4345
server  client  [TCP segment of a reassembled PDU]
client  server  [RST] seq=221
server  client  Continuation or non-HTTP traffic
client  server  [RST] seq=221

In my limited understanding, FIN means 'I'm done sending', which IMO is fair enough since the client headers are already sent. However RST/reset seems to just attempt to drop the connection.

在我有限的理解中，FIN 的意思是 'I'm donesent '，IMO 足够公平，因为客户端标头已经发送。然而，RST/reset 似乎只是试图断开连接。

The HTTP headers for client are:

客户端的 HTTP 标头是：

GET /some/path?params=values HTTP/1.1
connection: Keep-Alive
host: target.host.com
accept: */*
user-agent: Wget/1.14 (linux-gnu)

And for the server:

对于服务器：

HTTP/1.1 200 OK
Date: Mon, 16 Sep 2013 03:59:37 GMT
Server: Apache-Coyote/1.1
Content-Disposition: inline; filename=geoserver-GetFeature.text
Content-Type: text/xml; subtype=gml/2.1.2
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked

btw, this question: [restlet ]TruncatedChunkException:looks similar, but doesn't seem to have any helpful info.

顺便说一句，这个问题：[restlet ]TruncatedChunkException:看起来很相似，但似乎没有任何有用的信息。

Update:I've tried with a non-chunked site (/. :-) ) and it fails similarly with a:

更新：我已经尝试过使用非分块站点 (/. :-) )，但同样失败了：

org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body

采纳答案by Phasmal

OK, I found out what I'd done. I'd cleaned up my connection too early - basically the boilerplate in my connection method had a

好的，我发现我做了什么。我太早清理了我的连接 - 基本上我的连接方法中的样板有一个

finally
{
    client.getConnectionManager().shutdown();
}

but the method returned the stream object, so reading wasn't completed when the shutdown occurred.

但是该方法返回了流对象，因此关闭发生时读取未完成。