如何保持多个 Java HttpConnections 打开到同一目的地

how to keep multiple Java HttpConnections open to same destination

提问by Lightbeard

We are using HttpURLConnection API to invoke a REST API to the same provider often (kind of an aggregation usecase). We want to keep a pool of 5 connections always open to the provider host (always the same IP).

我们经常使用 HttpURLConnection API 向同一个提供者调用 REST API（一种聚合用例）。我们希望保持 5 个连接池始终对提供者主机开放（始终使用相同的 IP）。

What is the proper solution? Here is what we tried:

什么是正确的解决方案？这是我们尝试过的：

System.setProperty("http.maxConnections", 5);  // set globally only once
...
// everytime we need a connection, we use the following
HttpURLConnection conn = (HttpURLConnection) (new URL(url)).openConnection();
conn.setRequestMethod("GET");
conn.setDoInput(true);
conn.setDoOutput(false);
conn.setUseCaches(true);
...
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
...

At this point we read the input stream until the BufferedReader returns no more bytes. What do we do after that point if we want to reuse the underlying connection to the provider? We were under the impression that if the input stream is completely read, the connection is then added back to the pool.

此时我们读取输入流，直到 BufferedReader 不再返回字节。如果我们想重用与提供者的底层连接，在那之后我们该怎么办？我们的印象是，如果输入流被完全读取，则连接会被添加回池中。

It's been working for several weeks this way, but today it stopped working producing this exception: java.net.SocketException: Too many open files

它已经以这种方式工作了几个星期，但今天它停止工作，产生了这个异常： java.net.SocketException: Too many open files

We found numerous sockets in the CLOSE_WAIT state like this (by running lsof): java 1814 root 97u IPv6 844702 TCP colinux:58517->123.123.254.205:www (CLOSE_WAIT)

我们在 CLOSE_WAIT 状态中发现了许多这样的套接字（通过运行lsof）： java 1814 root 97u IPv6 844702 TCP colinux:58517->123.123.254.205:www (CLOSE_WAIT)

Won't either conn.getInputStream().close() or conn.disconnect() completely close the connection and remove it from the pool?

conn.getInputStream().close() 或 conn.disconnect() 不会完全关闭连接并将其从池中删除吗？

采纳答案by Alexander Torstling

From here:

从这里：

The current implementation doesn't buffer the response body. Which means that the application has to finish reading the response body or call close() to abandon the rest of the response body, in order for that connection to be reused. Furthermore, current implementation will not try block-reading when cleaning up the connection, meaning if the whole response body is not available, the connection will not be reused.

当前实现不缓冲响应正文。这意味着应用程序必须完成读取响应正文或调用 close() 以放弃响应正文的其余部分，以便重用该连接。此外，当前的实现在清理连接时不会尝试块读取，这意味着如果整个响应主体不可用，连接将不会被重用。

I read this as if your solution should work, but that you are also free to call close and the connection will still be reused.

我读到这就像您的解决方案应该有效，但您也可以自由调用 close 并且连接仍将被重用。

回答by ZZ Coder

We had this problem also on Java 5 and our solution is to switch to Apache HttpClient with pooled connection manager.

我们在 Java 5 上也遇到了这个问题，我们的解决方案是使用池化连接管理器切换到 Apache HttpClient。

The keepalive implementation of Sun's URL handler for HTTP is very buggy. There is no maintenance thread to close idle connections.

Sun 的 HTTP 的 URL 处理程序的 keepalive 实现有很多问题。没有维护线程来关闭空闲连接。

Another bigger problem with keepalive is that you need to delete responses. Otherwise, the connection will be orphaned also. Most people don't handle error stream correctly. Please see my answer to this question for an example on how to read error responses correctly,

keepalive 的另一个更大的问题是您需要删除响应。否则，连接也将是孤立的。大多数人没有正确处理错误流。有关如何正确读取错误响应的示例，请参阅我对此问题的回答，

HttpURLConnection.getResponseCode() returns -1 on second invocation

HttpURLConnection.getResponseCode() 在第二次调用时返回 -1

回答by Lightbeard

The referencecited by disown was what really helped.

disown 引用的参考文献真正有帮助。

We know Apache HttpClient is better, but that would require another jar and we might use this code in an applet.

我们知道 Apache HttpClient 更好，但这需要另一个 jar，我们可能会在小程序中使用此代码。

Calling HttpURLConnection.connect()was unnecessary. I'm not sure if it prevents connection reuse, but we took it out. It is safe to close the stream, but calling disconnect()on the connection will prevent reuse. Also, setting sun.net.http.errorstream.enableBuffering=truehelps.

打电话HttpURLConnection.connect()是不必要的。我不确定它是否会阻止连接重用，但我们将其删除。关闭流是安全的，但调用disconnect()连接会阻止重用。此外，设置有sun.net.http.errorstream.enableBuffering=true帮助。

Here is what we ended up using:

这是我们最终使用的：

System.setProperty("http.maxConnections", String.valueOf(CONST.CONNECTION_LIMIT));
System.setProperty("sun.net.http.errorstream.enableBuffering", "true");

...

int responseCode = -1;
HttpURLConnection conn = null;
BufferedReader reader = null;
try {
 conn = (HttpURLConnection) (new URL(url)).openConnection();
 conn.setRequestProperty("Accept-Encoding", "gzip");

 // this blocks until the connection responds
 InputStream in = new GZIPInputStream(conn.getInputStream());

 reader = new BufferedReader(new InputStreamReader(in));
 StringBuffer sb = new StringBuffer();
 char[] buff = new char[CONST.HTTP_BUFFER_SIZE];
 int cnt;

 while((cnt = reader.read(buff)) > 0) sb.append(buff, 0, cnt);

 reader.close();

 responseCode = conn.getResponseCode();
 if(responseCode != HttpURLConnection.HTTP_OK) throw new IOException("abnormal HTTP response code:"+responseCode);

 return sb.toString();

} catch(IOException e) {
    // consume error stream, otherwise, connection won't be reused
    if(conn != null) {
     try {
         InputStream in = ((HttpURLConnection)conn).getErrorStream();
         in.close();
         if(reader != null) reader.close();
     } catch(IOException ex) {
         log.fine(ex);
     }
    }

    // log exception    
    String rc = (responseCode == -1) ? "unknown" : ""+responseCode;
    log.severe("Error for HttpUtil.httpGet("+url+")\nServer returned an HTTP response code of '"+rc+"'");
    log.severe(e);
}