java 发送 HTTP 时的 OutputStream OutOfMemoryError
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/2082057/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
OutputStream OutOfMemoryError when sending HTTP
提问by ashchawla
I am trying to publish a large video/image file from the local file system to an http path, but I run into an out of memory error after some time...
我正在尝试将大型视频/图像文件从本地文件系统发布到 http 路径，但一段时间后我遇到了内存不足错误...
here is the code
这是代码
public boolean publishFile(URI publishTo, String localPath) throws Exception {
InputStream istream = null;
OutputStream ostream = null;
boolean isPublishSuccess = false;
URL url = makeURL(publishTo.getHost(), this.port, publishTo.getPath());
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
if (conn != null) {
try {
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("PUT");
istream = new FileInputStream(localPath);
ostream = conn.getOutputStream();
int n;
byte[] buf = new byte[4096];
while ((n = istream.read(buf, 0, buf.length)) > 0) {
ostream.write(buf, 0, n); //<--- ERROR happens on this line.......???
}
int rc = conn.getResponseCode();
if (rc == 201) {
isPublishSuccess = true;
}
} catch (Exception ex) {
log.error(ex);
} finally {
if (ostream != null) {
ostream.close();
}
if (istream != null) {
istream.close();
}
}
}
return isPublishSuccess;
}
HEre is the error i am getting...
这是我遇到的错误...
Exception in thread "Thread-8773" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
at com.test.HTTPClient.publishFile(HTTPClient.java:110)
at com.test.HttpFileTransport.put(HttpFileTransport.java:97)
回答by kdgregory
The HttpUrlConnectionis buffering the data so that it can set the Content-Lengthheader (per HTTP spec).
所述HttpUrlConnection被缓冲的数据，以便它可以设置Content-Length（每头HTTP规范）。
One alternative, if your destination server supports it, is to use "chunked" transfers. This will buffer only a small portion of data at a time. However, not all services support it (Amazon S3, for example, doesn't).
如果您的目标服务器支持，另一种选择是使用“分块”传输。这一次只会缓冲一小部分数据。但是，并非所有服务都支持它（例如，Amazon S3 不支持）。
Another alternative (and imo a better one) is to use Jakarta HttpClient. You can set the "entity" in a request from a file, and the connection code will set request headers appropriately.
另一种选择（imo 更好）是使用Jakarta HttpClient。您可以在来自文件的请求中设置“实体”，连接代码将适当地设置请求标头。
Edit: noscommented that the OP could call HttpURLConnection.setFixedLengthStreamingMode(long length). I was unaware of this method; it was added in 1.5, and I haven't used this class since then.
编辑：nos评论说 OP 可以调用HttpURLConnection.setFixedLengthStreamingMode(long length). 我不知道这种方法；它是在 1.5 中添加的，从那时起我就没有使用过这个类。
However, I stillsuggest using Jakarta HttpClient, for the simple reason that it reduces the amount of code that the OP has to maintain. Code that is boilerplate, yet still has the potential for errors:
但是，我仍然建议使用 Jakarta HttpClient，原因很简单，它减少了 OP 必须维护的代码量。样板代码，但仍有可能出错：
- The OP correctly handles the loop to copy between input and output. Usually when I see an example of this, the poster either doesn't properly check the returned buffer size, or keeps re-allocating the buffers. Congratulations, but you now have to ensure that your successors take as much care.
- The exception handling isn't quite so good. Yes, the OP remembers to close the connections in a
finallyblock, and again, congratulations on that. Except that either of theclose()calls could throwIOException, keeping the other from executing. And the method as a whole throwsException, so that the compiler isn't going to help catch similar errors. - I count 31 lines of code to setup and execute the response (excluding the response code check and the URL computation, but including the try/catch/finally). With HttpClient, this would be somewhere in the range of a half dozen LOC.
- OP 正确处理循环以在输入和输出之间复制。通常当我看到这样的例子时，发布者要么没有正确检查返回的缓冲区大小，要么不断重新分配缓冲区。恭喜，但您现在必须确保您的继任者尽可能小心。
- 异常处理不是很好。是的，OP 记得在一个
finally块中关闭连接，再次祝贺你。除了任何一个close()调用都可能 throwIOException，使另一个无法执行。并且该方法作为一个整体 throwsException，因此编译器不会帮助捕获类似的错误。 - 我计算了 31 行代码来设置和执行响应（不包括响应代码检查和 URL 计算，但包括 try/catch/finally）。使用 HttpClient，这将在半打 LOC 范围内的某个地方。
Even if the OP had written this code perfectly, and refactored it into methods similar to those in Jakarta Commons IO, s/he shouldn't do that. This code has been written and tested by others. I know that it's a waste of mytime to rewrite it, and suspect that it's a waste of the OP's time as well.
即使 OP 完美地编写了这段代码，并将其重构为类似于 Jakarta Commons IO 中的方法，他/她也不应该那样做。此代码已由其他人编写和测试。我知道重写它是在浪费我的时间，并且怀疑这也是在浪费 OP 的时间。
回答by Alexandr
conn.setFixedLengthStreamingMode((int) new File(localpath).length());
And for buffering you could cover your streams into the BufferedOutputStreamand BufferedInputStream
对于缓冲，您可以将流覆盖到BufferedOutputStream和BufferedInputStream 中
Good example of chunked uploading you could find there: gdata-java-client
你可以在那里找到分块上传的好例子：gdata-java-client
回答by Piyush Chordia
HttpsURLConnection#setChunkedStreamingMode(1024 * 1024 * 10); //10MB chunk This ensures that any file (of any size) is streamed over a https connection, without internal buffering. This should be used when the file size or the content length is unknown.
HttpsURLConnection#setChunkedStreamingMode(1024 * 1024 * 10); //10MB 块 这确保任何文件（任何大小）都通过 https 连接流式传输，没有内部缓冲。当文件大小或内容长度未知时，应该使用它。
回答by Chris Dail
The problem is that the HttpURLConnection class is using a byte array to store your data. Presumably this video you are pushing is taking more memory than available. You have a few options here:
问题是 HttpURLConnection 类使用字节数组来存储您的数据。据推测，您正在推送的此视频占用的内存超过可用内存。您在这里有几个选择：
Increase the memory to your application. You can use the -Xmx1024m option to give 1GB of memory to your application. This will increase the amount of data you can store in memory.
If you still run out of memory, you might want to consider trying another library to push the video up that does not store the data all in memory at once. The Apache Commons HttpClient has such a feature. See this site for more information: http://hc.apache.org/httpclient-3.x/features.html. See this section for multi-part form upload of large files: http://hc.apache.org/httpclient-3.x/methods/multipartpost.html
增加应用程序的内存。您可以使用 -Xmx1024m 选项为您的应用程序提供 1GB 内存。这将增加您可以存储在内存中的数据量。
如果您的内存仍然不足，您可能需要考虑尝试使用另一个库来推送视频，该库不会一次将数据全部存储在内存中。Apache Commons HttpClient 就有这样的功能。有关更多信息，请参阅此站点：http: //hc.apache.org/httpclient-3.x/features.html。有关大文件的多部分表单上传，请参阅本节：http: //hc.apache.org/httpclient-3.x/methods/multipartpost.html
回答by skaffman
For anything other than basic GET operations, the built-in java.netHTTP stuff isn't very good. Using Apache Commons HttpClientis recommended for this. It lets you do much more intuitive stuff like this:
对于基本 GET 操作以外的任何操作，内置的java.netHTTP 内容都不是很好。为此，建议使用Apache Commons HttpClient。它可以让你做更直观的事情，比如：
PutMethod put = new PutMethod(url);
put.setRequestEntity(new FileRequestEntity(localFile, contentType));
int responseCode = put.executeMethod();
which replaces a lot of your boiler-plate code.
它取代了很多样板代码。
回答by duffymo
Your problem is that you're trying to fix X video bytes into X/N bytes of RAM, when N > 1.
您的问题是当 N > 1 时，您试图将 X 个视频字节修复到 X/N 个字节的 RAM 中。
You either need to read the video into a smaller buffer and write it out as you go or make the file smaller or increase the memory available to your process.
您需要将视频读入较小的缓冲区并随时将其写出，或者缩小文件或增加进程可用的内存。
Check your heap size. You can use -Xmx to increase it if you've taken the default.
检查您的堆大小。如果您采用默认值，则可以使用 -Xmx 来增加它。
