Java 使用 RestTemplate 获取 InputStream

Getting InputStream with RestTemplate

提问by user1950349

I am using URL class to read an InputStream from it. Is there any way I can use RestTemplate for this?

我正在使用 URL 类从中读取 InputStream。有什么办法可以为此使用 RestTemplate 吗？

InputStream input = new URL(url).openStream();
JsonReader reader = new JsonReader(new InputStreamReader(input, StandardCharsets.UTF_8.displayName())); 

How can I get InputStreamwith RestTemplateinstead of using URL?

我怎样才能InputStream使用RestTemplate而不是使用URL？

采纳答案by Sotirios Delimanolis

You should not get the InputStreamdirectly. RestTemplateis meant to encapsulate processing the response (and request) content. Its strength is handling all the IO and handing you a ready-to-go Java object.

你不应该InputStream直接得到。RestTemplate旨在封装处理响应（和请求）内容。它的优势在于处理所有 IO 并为您提供一个现成的 Java 对象。

You'll need to register appropriate HttpMessageConverterobjects. Those will have access to the response's InputStream, through an HttpInputMessageobject.

您需要注册适当的HttpMessageConverter对象。那些将可以InputStream通过HttpInputMessage对象访问响应的, 。

As Abdull suggests, Spring does come with an HttpMessageConverterimplementation for Resourcewhich itself wraps an InputStream, ResourceHttpMessageConverter. It doesn't support all Resourcetypes, but since you should be programming to interfaces anyway, you should just use the superinterface Resource.

正如 Abdull 所建议的那样，Spring 确实带有一个HttpMessageConverter实现，Resource它本身包装了一个InputStream, ResourceHttpMessageConverter。它不支持所有Resource类型，但由于您无论如何都应该对接口进行编程，因此您应该只使用 superinterface Resource。

The current implementation (4.3.5), will return a ByteArrayResourcewith the content of the response stream copied to a new ByteArrayInputStreamwhich you can access.

当前的实现（4.3.5）将返回一个ByteArrayResource响应流的内容复制到一个ByteArrayInputStream你可以访问的新的。

You don't have to close the stream. The RestTemplatetakes care of that for you. (This is unfortunate if you try to use a InputStreamResource, another type supported by the ResourceHttpMessageConverter, because it wraps the underlying response's InputStreambut is closed before it can be exposed to your client code.)

您不必关闭流。TheRestTemplate会为您处理这些。（如果您尝试使用InputStreamResource，这是不幸的，这是由 支持的另一种类型ResourceHttpMessageConverter，因为它包装了底层响应，InputStream但在它可以暴露给您的客户端代码之前已关闭。）

回答by inigo skimmer

As a variant you can consume response as bytes and than convert to stream

作为变体，您可以将响应作为字节使用，而不是转换为流

byte data[] = restTemplate.execute(link, HttpMethod.GET, null, new BinaryFileExtractor());
return new ByteArrayInputStream(data);

Extractor is

提取器是

public class BinaryFileExtractor implements ResponseExtractor<byte[]> {

  @Override
  public byte[] extractData(ClientHttpResponse response) throws IOException {
    return ByteStreams.toByteArray(response.getBody());
  }
}

回答by Abdull

Spring has a org.springframework.http.converter.ResourceHttpMessageConverter. It converts Spring's org.springframework.core.io.Resourceclass. That Resourceclass encapsulates a InputStream, which you can obtain via someResource.getInputStream().

春天有一个org.springframework.http.converter.ResourceHttpMessageConverter。它转换 Spring 的org.springframework.core.io.Resource类。那Resource类封装InputStream，您可以通过获得someResource.getInputStream()。

Putting this all together, you can actually get an InputStreamvia RestTemplateout-of-the-box by specifying Resource.classas your RestTemplateinvocation's response type.

将所有这些放在一起，您实际上可以通过指定为调用的响应类型来获得开箱即用的InputStreamvia 。RestTemplateResource.classRestTemplate

Here is an example using one of RestTemplate's exchange(..)methods:

这是使用RestTemplate的exchange(..)方法之一的示例：

import org.springframework.web.client.RestTemplate;
import org.springframework.http.HttpMethod;
import org.springframework.core.io.Resource;

ResponseEntity<Resource> responseEntity = restTemplate.exchange( someUrlString, HttpMethod.GET, someHttpEntity, Resource.class );

InputStream responseInputStream;
try {
    responseInputStream = responseEntity.getBody().getInputStream();
}
catch (IOException e) {
    throw new RuntimeException(e);
}

// use responseInputStream

回答by Abhijit Sarkar

The previous answers are not wrong, but they don't go into the depth that I like to see. There are cases when dealing with low level InputStreamis not only desirable, but necessary, the most common example being streaming a large file from source (some web server) to destination (a database). If you try to use a ByteArrayInputStream, you will be, not so surprisingly, greeted with OutOfMemoryError. Yes, you can roll your own HTTP client code, but you'll have to deal with erroneous response codes, response converters etc. If you are already using Spring, looking to RestTemplateis a natural choice.

前面的答案没有错，但没有深入到我喜欢看到的深度。在某些情况下，处理低级别InputStream不仅是可取的，而且是必要的，最常见的例子是将大文件从源（某些 Web 服务器）流式传输到目标（数据库）。如果您尝试使用ByteArrayInputStream，您会毫不奇怪地受到OutOfMemoryError. 是的，您可以使用自己的 HTTP 客户端代码，但是您必须处理错误的响应代码、响应转换器等。如果您已经在使用 Spring，那么寻找RestTemplate是一个自然的选择。

As of this writing, spring-web:5.0.2.RELEASEhas a ResourceHttpMessageConverterthat has a boolean supportsReadStreaming, which if set, and the response type is InputStreamResource, returns InputStreamResource; otherwise it returns a ByteArrayResource. So clearly, you're not the only one that asked for streaming support.

在撰写本文时，spring-web:5.0.2.RELEASEhas aResourceHttpMessageConverter有 a boolean supportsReadStreaming，如果设置了，并且响应类型为InputStreamResource，则返回InputStreamResource；否则它返回一个ByteArrayResource. 很明显，您并不是唯一一个要求流媒体支持的人。

However, there is a problem: RestTemplatecloses the response soon after the HttpMessageConverterruns. Thus, even if you asked for InputStreamResource, and got it, it's no good, because the response stream has been closed. I think this is a design flaw that they overlooked; it should've been dependent on the response type. So unfortunately, for reading, you must consume the response fully; you can't pass it around if using RestTemplate.

但是，有一个问题：运行RestTemplate后很快关闭响应HttpMessageConverter。因此，即使您要求InputStreamResource并得到它，也没有好处，因为响应流已关闭。我认为这是他们忽略的设计缺陷；它应该取决于响应类型。所以不幸的是，为了阅读，你必须完全消费响应；如果使用RestTemplate.

Writing is no problem though. If you want to stream an InputStream, ResourceHttpMessageConverterwill do it for you. Under the hood, it uses org.springframework.util.StreamUtilsto write 4096 bytes at a time from the InputStreamto the OutputStream.

不过写是没问题的。如果你想流式传输InputStream，ResourceHttpMessageConverter会为你做。在幕后，它使用org.springframework.util.StreamUtils从InputStream到一次写入 4096 个字节OutputStream。

Some of the HttpMessageConvertersupport all media types, so depending on your requirement, you may have to remove the default ones from RestTemplate, and set the ones you need, being mindful of their relative ordering.

有些HttpMessageConverter支持所有媒体类型，因此根据您的要求，您可能需要从 中删除默认的RestTemplate，并设置您需要的，注意它们的相对顺序。

Last but not the least, implementations of ClientHttpRequestFactoryhas a boolean bufferRequestBodythat you can, and should, set to falseif you are uploading a large stream. Otherwise, you know, OutOfMemoryError. As of this writing, SimpleClientHttpRequestFactory(JDK client) and HttpComponentsClientHttpRequestFactory(Apache HTTP client) support this feature, but not OkHttp3ClientHttpRequestFactory. Again, design oversight.

最后但并非最不重要的，实现的ClientHttpRequestFactory有boolean bufferRequestBody，你可以，也应该设置为false你上载大量流。否则，你知道，OutOfMemoryError。在撰写本文时，SimpleClientHttpRequestFactory（JDK 客户端）和HttpComponentsClientHttpRequestFactory（Apache HTTP 客户端）支持此功能，但不支持OkHttp3ClientHttpRequestFactory. 再次，设计监督。

Edit: Filed ticket SPR-16885.

编辑：已提交票证SPR-16885。

回答by Sebien

Thanks to Abhijit Sarkar's answer for leading the way.

感谢 Abhijit Sarkar 的回答带路。

I needed to download a heavy JSON stream and break it into small streamable manageable pieces of data. The JSON is composed of objects that have big properties: such big properties can be serialized to a file, and thus removed from the unmarshalled JSON object.

我需要下载一个沉重的 JSON 流并将其分解成小的可流式管理的数据块。JSON 由具有大属性的对象组成：这样的大属性可以序列化为文件，从而从未编组的 JSON 对象中删除。

Another use case is to download a JSON stream object by object, process it like a map/reduce algorythm and produce a single output without having to load the whole stream in memory.

另一个用例是逐个对象下载 JSON 流对象，像 map/reduce 算法一样处理它并生成单个输出，而无需将整个流加载到内存中。

Yet another use case is to read a big JSON file and only pick a few objects based on a condition, while unmarshalling to Plain Old Java Objects.

另一个用例是读取一个大的 JSON 文件并仅根据条件选择几个对象，同时解组为普通的旧 Java 对象。

Here is an example: we'd like to stream a very huge JSON file that is an array, and we'd like to retrieve only the first object in the array.

下面是一个例子：我们想流式传输一个非常大的 JSON 文件，它是一个数组，我们只想检索数组中的第一个对象。

Given this big file on a server, available at http://example.org/testings.json:

鉴于服务器上的这个大文件，可在http://example.org/testings.json 获得：

[
    { "property1": "value1", "property2": "value2", "property3": "value3" },
    { "property1": "value1", "property2": "value2", "property3": "value3" },
    ... 1446481 objects => a file of 104 MB => take quite long to download...
]

Each row of this JSON array can be parsed as this object:

这个 JSON 数组的每一行都可以解析为这个对象：

@lombok.Data
public class Testing {
    String property1;
    String property2;
    String property3;
}

You need this class make the parsing code reusable:

您需要这个类使解析代码可重用：

import com.fasterxml.Hymanson.core.JsonParser;
import java.io.IOException;
@FunctionalInterface
public interface JsonStreamer<R> {
    /**
     * Parse the given JSON stream, process it, and optionally return an object.<br>
     * The returned object can represent a downsized parsed version of the stream, or the result of a map/reduce processing, or null...
     *
     * @param jsonParser the parser to use while streaming JSON for processing
     * @return the optional result of the process (can be {@link Void} if processing returns nothing)
     * @throws IOException on streaming problem (you are also strongly encouraged to throw HttpMessageNotReadableException on parsing error)
     */
    R stream(JsonParser jsonParser) throws IOException;
}

And this class to parse:

而这个类要解析：

import com.fasterxml.Hymanson.core.JsonFactory;
import com.fasterxml.Hymanson.core.JsonParser;
import lombok.AllArgsConstructor;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;

import java.io.IOException;
import java.util.Collections;
import java.util.List;

@AllArgsConstructor
public class StreamingHttpMessageConverter<R> implements HttpMessageConverter<R> {

    private final JsonFactory factory;
    private final JsonStreamer<R> jsonStreamer;

    @Override
    public boolean canRead(Class<?> clazz, MediaType mediaType) {
        return MediaType.APPLICATION_JSON.isCompatibleWith(mediaType);
    }

    @Override
    public boolean canWrite(Class<?> clazz, MediaType mediaType) {
        return false; // We only support reading from an InputStream
    }

    @Override
    public List<MediaType> getSupportedMediaTypes() {
        return Collections.singletonList(MediaType.APPLICATION_JSON);
    }

    @Override
    public R read(Class<? extends R> clazz, HttpInputMessage inputMessage) throws IOException {
        try (InputStream inputStream = inputMessage.getBody();
             JsonParser parser = factory.createParser(inputStream)) {
            return jsonStreamer.stream(parser);
        }
    }

    @Override
    public void write(R result, MediaType contentType, HttpOutputMessage outputMessage) {
        throw new UnsupportedOperationException();
    }

}

Then, here is the code to use to stream the HTTP response, parse the JSON array and return only the first unmarshalled object:

然后，这里是用于流式传输 HTTP 响应、解析 JSON 数组并仅返回第一个未编组对象的代码：

// You should @Autowire these:
JsonFactory jsonFactory = new JsonFactory();
ObjectMapper objectMapper = new ObjectMapper();
RestTemplateBuilder restTemplateBuilder = new RestTemplateBuilder();

// If detectRequestFactory true (default): HttpComponentsClientHttpRequestFactory will be used and it will consume the entire HTTP response, even if we close the stream early
// If detectRequestFactory false: SimpleClientHttpRequestFactory will be used and it will close the connection as soon as we ask it to

RestTemplate restTemplate = restTemplateBuilder.detectRequestFactory(false).messageConverters(
    new StreamingHttpMessageConverter<>(jsonFactory, jsonParser -> {

        // While you use a low-level JsonParser to not load everything in memory at once,
        // you can still profit from smaller object mapping with the ObjectMapper
        if (!jsonParser.isClosed() && jsonParser.nextToken() == JsonToken.START_ARRAY) {
            if (!jsonParser.isClosed() && jsonParser.nextToken() == JsonToken.START_OBJECT) {
                return objectMapper.readValue(jsonParser, Testing.class);
            }
        }
        return null;

    })
).build();

final Testing firstTesting = restTemplate.getForObject("http://example.org/testings.json", Testing.class);
log.debug("First testing object: {}", firstTesting);

回答by neesh

You can pass in your own response extractor. Here is an example where I write out the json to disk in a streaming fashion -

您可以传入自己的响应提取器。这是一个示例，我以流式方式将 json 写入磁盘 -

        RestTemplate restTemplate = new RestTemplateBuilder().basicAuthentication("user", "their_password" ).build();

        int responseSize = restTemplate.execute(uri,
            HttpMethod.POST,
            (ClientHttpRequest requestCallback) -> {
                requestCallback.getHeaders().setContentType(MediaType.APPLICATION_JSON);
                requestCallback.getBody().write(body.getBytes());
            },
            responseExtractor -> {
                FileOutputStream fos  = new FileOutputStream(new File("out.json"));
                return StreamUtils.copy(responseExtractor.getBody(), fos);
            }
    )

回答by ItamarBe

I encountered the same issue and solved it by extending RestTemplate and closing the connection only after the stream is read.

我遇到了同样的问题，并通过扩展 RestTemplate 并仅在读取流后关闭连接来解决它。

you can see the code here: https://github.com/ItamarBenjamin/stream-rest-template

你可以在这里看到代码：https: //github.com/ItamarBenjamin/stream-rest-template