Java 如何处理来自 REST 服务的大量数据
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/21608974/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to handle huge data from a REST service
提问by om39a
We are using a REST service that returns huge data. In production, the server hardware can handle it. I need to test the REST service by getting the data on my local machine, which cannot handle the huge data. My local machine is a 4G i5. I am getting out of memory exception every time I hit the service.
我们正在使用返回大量数据的 REST 服务。在生产中，服务器硬件可以处理它。我需要通过获取本地机器上的数据来测试 REST 服务，该机器无法处理大量数据。我的本地机器是 4G i5。每次点击服务时，我都会出现内存不足的异常。
response.getStatus()returns 200 status.
But while collecting the data using input stream reader, I get an out of memory exception.
response.getStatus()返回 200 状态。但是在使用输入流读取器收集数据时，出现内存不足异常。
BufferedReader br = new BufferedReader(new
InputStreamReader(newByteArrayInputStream(response.getEntity().getBytes())));
Is there any other way to collect the data without hitting the memory exception?
有没有其他方法可以在不碰到内存异常的情况下收集数据？
I tried pushing my VM size to 2G but, it still doesn't work.
我尝试将我的 VM 大小推到 2G，但它仍然不起作用。
采纳答案by Brian Kelly
If the service supports it, use the HTTP standard approach of requesting chunkswithin the range of the entire response content payload.
如果服务支持它，请使用 HTTP 标准方法请求整个响应内容有效负载范围内的块。
Another approach (again, assuming the API supports it) is pagination.
另一种方法（同样，假设 API 支持它）是分页。
回答by DwB
You can implement a threshold setting in the REST service configuration that limits the amount of data returned. In production, the threshold would either not be set or would be set to a large value. In your test environment, you can set the threshold to a small, managable value.
您可以在 REST 服务配置中实施阈值设置，以限制返回的数据量。在生产中，阈值要么不设置，要么设置为一个很大的值。在您的测试环境中，您可以将阈值设置为一个小的、可管理的值。
