Java 如何使用 apache HttpClient 流式传输响应正文

How to stream response body with apache HttpClient

提问by Dr.Knowitall

There's an api I need to perform octet-streaming from which does not have a length. It is just a stream of real time data. The issue that I'm having is that when I make my request, it seems to try to wait out for the end of the content before reading information into the inputstream, however it's not seeing the end of the content and timingout with NoHttpResponse exception. Below is a simplified version of my code:

我需要一个 api 来执行没有长度的八位字节流。它只是一个实时数据流。我遇到的问题是，当我提出请求时，它似乎在将信息读入输入流之前尝试等待内容结束，但是它没有看到内容结束和 NoHttpResponse 异常超时。下面是我的代码的简化版本：

private static HttpPost getPostRequest() {
    // Build uri
    URI uri = new URIBuilder()
            .setScheme("https")
            .setHost(entity.getStreamUrl())
            .setPath("/")
            .build();

    // Create http http
    HttpPost httpPost = new HttpPost(uri);

    String nvpsStr = "";
    Object myArray[] = nvps.toArray();
    for(int i = 0; i < myArray.length; i ++) {
        nvpsStr += myArray[i].toString();
        if(i < myArray.length - 1) {
            nvpsStr += "&";
        }
    }

    // Build http payload
    String request = nvpsStr + scv + streamRequest + "\n\n";
    // Attach http data
    httpPost.setEntity(new StringEntity(URLEncoder.encode(request,"UTF-8")));

    return httpPost;
}

// Where client is simply
// private static final CloseableHttpClient client = HttpClients.createDefault();
private static runPostRequest (HttpPost request) {
    CloseableHttpResponse response = client.execute(request);
    try {
        HttpEntity ent = response.getEntity();
        InputStream is = ent.getContent();
        DataInputStream dis = new DataInputStream(is);
        // Only stream the first 200 bytes
        for(int i = 0; i < 200; i++) {
            System.out.println(( (char)dis.readByte()));
        }

    } finally {
        response.close();
    }
}

采纳答案by Robert Rowntree

EDIT 2

编辑 2

So, if you're not comfortable with threads/runnables/Handlers and not comfortable with android AsyncTask, I would just go straight to HttpUrlConnection (drop the entire excercise with Apache HttpClient because, basically Google says that HttpUrlConnection will support streamed response and it does work!)

因此，如果您对线程/可运行程序/处理程序不满意并且对 android AsyncTask 不满意，我会直接转到 HttpUrlConnection（使用 Apache HttpClient 放弃整个练习，因为基本上谷歌说 HttpUrlConnection 将支持流式响应，并且确实如此工作！）

It may not be as easy instrumenting all the details like dumping headers. But with a normal streamed response Object, I think that it should just work.... see edit 3 for HttpsUrlConnection code sample

检测所有细节（如转储标头）可能并不容易。但是对于普通的流式响应对象，我认为它应该可以正常工作......请参阅 HttpsUrlConnection 代码示例的编辑 3

EndEdit2

结束编辑2

Not clear from the question what 'stream' protocol is being used (progressive download or HTTP streaming) OR how you are actually managing the streamed response on your client.

从问题中不清楚正在使用什么“流”协议（渐进式下载或 HTTP 流）或您如何实际管理客户端上的流响应。

Recommended to dump the headers from the connection to see exactly what the client and server are agreeing on??

建议从连接中转储标头以查看客户端和服务器到底同意什么？

I'm assuming that you are OFF the UI thread (either in AsyncTask or in the callback portion of a Handler); if that's not accurate you may have to refactor a little bit.

我假设您已关闭 UI 线程（在 AsyncTask 中或在 Handler 的回调部分）；如果这不准确，您可能需要稍微重构一下。

Assuming HTTP stream in use with Apache HttpClient 4.3.5+

假设 HTTP 流与 Apache HttpClient 4.3.5+ 一起使用

If there is no length in the headers of the response, then you are doing a 'chunked' response on HTTP 1.1 where you have to read a buffer until you get a 'last-chunk' or decide to CLOSE either the stream or the Connection:

如果响应的标头中没有长度，那么您正在 HTTP 1.1 上执行“分块”响应，您必须读取缓冲区，直到获得“最后一个块”或决定关闭流或连接：

The server just starts sending (streaming) and the client should handle the 'input-stream' that it gets from the HTTP response by employing a buffer as per the detailed Apache notes on producing entity content.

服务器刚刚开始发送（流式传输），客户端应该根据 Apache生成实体内容的详细说明使用缓冲区来处理它从 HTTP 响应中获取的“输入流” 。

I don't remember offhand if socket timeout of 30 seconds will pre-empt an active stream? Remember in Apache, separate settings exist in the builder for socket timeout, and readtimeout. Don't want socket to close on you and don't want to timeout waiting on a readable stream's available bytes while server is providing the response.

我不记得 30 秒的套接字超时是否会抢占活动流？请记住，在 Apache 中，构建器中存在单独的套接字超时设置和读取超时设置。不希望套接字在您身上关闭，也不希望在服务器提供响应时超时等待可读流的可用字节。

Anyway, the client-side handler just needs to be aware of how the stream ends by inspection of what's read into the buffer...

无论如何，客户端处理程序只需要通过检查读入缓冲区的内容来了解​​流如何结束......

If the protocol in place is "continue" & "chunked" then the response handler on the client should be in a stream handler loop until it sees the LAST-CHUNK from the http spec.

如果适当的协议是“继续”和“分块”，那么客户端上的响应处理程序应该处于流处理程序循环中，直到它看到http 规范中的 LAST-CHUNK 为止。

 response.getEntity().getContent() 

should give you the reference you need to process the response's stream until 'last-chunk'...

应该为您提供处理响应流所需的参考，直到“最后一个块”...

I think u should read hereon how to consume a buffered entity where more than a single read is going to be require to wind up at the 'last-chunk' in the response. It's another reason why HttpURLConnection may be easier...

我认为您应该在这里阅读有关如何使用缓冲实体的信息，其中需要不止一次读取才能在响应中的“最后一个块”处结束。这是 HttpURLConnection 可能更容易的另一个原因......

Do a loop that handles buffered reads until END signaled by the bytes matching 'last-chunk'.

执行一个循环处理缓冲读取，直到匹配“last-chunk”的字节发出 END 信号。

Then close either the stream or connection as per the detailed Apache notes on consuming entities and reusable Connections.

然后根据有关使用实体和可重用连接的详细 Apache 说明关闭流或连接。

EDITcode for streamed response in apache HttpClient

Apache HttpClient 中流式响应的编辑代码

In a 'handler's callback or in asyncTask

在“处理程序的回调”或 asyncTask 中

 request.execute();
...

 processStreamingEntity(response.getEntity());
 response.close();

//implement your own wrapper as mentioned in apache docs

    private void processStreamingEntity(HttpEntity entity) throws IOException {
        InputStreamHttpEntityHC4 bufHttpEntity = new InputStreamHttpEntityHC4(entity);
        while not bufHttpEntity.LAST_CHUNK {
            handleResponse(bufHttpEntity.readLine())
}

EDIT 3

编辑 3

HttpURLConnection version if you go that way. ( uses a MessageHandler but you could consume the bytes in place as this is from a streaming speach example and the words from text are being sent back to UI here)

HttpURLConnection 版本，如果你走那条路。（使用 MessageHandler 但您可以就地使用字节，因为这是来自流式语音示例，并且文本中的单词将被发送回 UI 此处）

private void openHttpsConnection(String urlStr, Handler mhandler) throws IOException {
    HttpsURLConnection httpConn = null;
    String line = null;
    try {
        URL url = new URL(urlStr);
        URLConnection urlConn = url.openConnection();               
        if (!(urlConn instanceof HttpsURLConnection)) {
            throw new IOException ("URL is not an Https URL");
        }               
        httpConn = (HttpsURLConnection)urlConn;
        httpConn.setAllowUserInteraction(false);
        httpConn.setInstanceFollowRedirects(true);
        httpConn.setRequestMethod("GET");
        httpConn.setReadTimeout(50 * 1000);
        BufferedReader is =
                new BufferedReader(new InputStreamReader(httpConn.getInputStream()));                   

        while ((line = is.readLine( )) != null) {

                Message msg = Message.obtain();
                msg.what=1;  
                msg.obj=line;                       
                mhandler.sendMessage(msg);

        }               
    } catch (MalformedURLException e) {
        e.printStackTrace();
    } catch( SocketTimeoutException e){
        e.printStackTrace();
    } catch (IOException e) {

        e.printStackTrace();
        Message msg = Message.obtain();
            msg.what=2;
            BufferedInputStream in = new BufferedInputStream(httpConn.getErrorStream());

            line =new String(readStream(in));
            msg.obj=line;
            mhandler.sendMessage(msg);

    }
    finally {httpConn.disconnect();}

}