java Android：在不耗尽内存的情况下将流转换为字符串
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/4729188/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Android: Convert Stream to String without running out of memory
提问by esilver
I have an android client that communicates with the server via REST-ful endpoints and JSON. Because of this, I have the need to retrieve the full server response prior to converting it into a Hash. I have this code in place to do that (found on the internet someplace):
我有一个 android 客户端，它通过 REST-ful 端点和 JSON 与服务器进行通信。因此，我需要在将其转换为哈希之前检索完整的服务器响应。我有这个代码来做到这一点（在互联网上的某个地方找到）：
private static String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
The code works for the most part, however I am seeing reports of crashes in the field from clients with an OutOfMemory exception on the line:
该代码在大多数情况下都有效，但是我看到来自客户端的现场崩溃报告，并在线上出现 OutOfMemory 异常：
while ((line = reader.readLine()) != null) {
The full stack trace is:
完整的堆栈跟踪是：
java.lang.RuntimeException: An error occured while executing doInBackground()
at android.os.AsyncTask.done(AsyncTask.java:200)
at java.util.concurrent.FutureTask$Sync.innerSetException(FutureTask.java:273)
at java.util.concurrent.FutureTask.setException(FutureTask.java:124)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:307)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561)
at java.lang.Thread.run(Thread.java:1102)
Caused by: java.lang.OutOfMemoryError
at java.lang.String.(String.java:468)
at java.lang.AbstractStringBuilder.toString(AbstractStringBuilder.java:659)
at java.lang.StringBuilder.toString(StringBuilder.java:664)
at java.io.BufferedReader.readLine(BufferedReader.java:448)
at com.appspot.myapp.util.RestClient.convertStreamToString(RestClient.java:303)
at com.appspot.myapp.util.RestClient.executeRequest(RestClient.java:281)
at com.appspot.myapp.util.RestClient.Execute(RestClient.java:178)
at com.appspot.myapp.$LoadProfilesTask.doInBackground(GridViewActivity.java:1178)
at com.appspot.myapp.$LoadProfilesTask.doInBackground(GridViewActivity.java:1)
at android.os.AsyncTask.call(AsyncTask.java:185)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
... 4 more
My question: is there any way to solve this problem aside from sending smaller chunks of data from the server?
我的问题：除了从服务器发送较小的数据块之外，还有什么方法可以解决这个问题？
Thanks!
谢谢！
采纳答案by Konstantin Komissarchik
In general, the answer is no, but you can certainly tune the conditions that cause you to run out of memory. In particular, if you send string length ahead of your stream, you will be able to create a StringBuilder with correct array size inside of it. Arrays cannot be resized after creation, so if you run out of array capacity in StringBuilder, the implementation has to allocate a new array (typically twice the size to avoid too many resizes) and then copy the old array contents. Consider stream of size X, to resize StringBuilder that just happened to be a X-1 capacity, you need almost X*3 amount of memory. Sizing StringBuilder such that resizes are avoided will allow you squeeze larger streams into memory.
一般来说，答案是否定的，但您当然可以调整导致内存不足的条件。特别是，如果您在流之前发送字符串长度，您将能够在其中创建一个具有正确数组大小的 StringBuilder。数组在创建后无法调整大小，因此如果 StringBuilder 中的数组容量用完，则实现必须分配一个新数组（通常是大小的两倍，以避免调整大小过多），然后复制旧数组内容。考虑大小为 X 的流，要调整刚好是 X-1 容量的 StringBuilder 的大小，您需要几乎 X*3 的内存量。调整 StringBuilder 的大小以避免调整大小将允许您将更大的流压缩到内存中。
Another thing you may want to do is to tune the amount of memory available to your server process. Use a switch like -Xmx1024m when launching the server process.
您可能想做的另一件事是调整服务器进程可用的内存量。启动服务器进程时使用 -Xmx1024m 之类的开关。
Of course, it would be much better to revise your algorithm to not require the entire stream to be held in memory. It will enable you to handle more clients with the same amount of hardware.
当然，最好修改您的算法以不要求将整个流保存在内存中。它将使您能够使用相同数量的硬件处理更多客户端。
回答by JLund
Android has restrictions on the maximum amount of memory you can allocate to your application. You could consider reading the stream on the flyand not save the whole thing into a String if the response is very large. But you should consider following best practice.
Android 对您可以分配给应用程序的最大内存量有限制。如果响应非常大，您可以考虑即时读取流，而不是将整个内容保存到 String 中。但是您应该考虑遵循最佳实践。
You should store the data in a sqlite database or to a regular file. It's not best practice to do what you're doing as the user might hit the home button or receive a phone call when you're in the middle of saving the response. It is better to use a database so that you can come back to the state where you were interrupted. Then you also do not have to worry about running out of memory.
您应该将数据存储在 sqlite 数据库或常规文件中。执行您正在执行的操作不是最佳做法，因为当您正在保存响应时，用户可能会按下主页按钮或接听电话。最好使用数据库，以便您可以回到被中断的状态。这样您也不必担心内存不足。
Have you watched this talk on best practices for communicating with REST services from Android? http://www.youtube.com/watch?v=xHXn3Kg2IQE?8m50s(8:50 and 11:20). Highly recommended to clear out best practices and why one should not retrieve REST data without using a database.
您是否看过有关与来自 Android 的 REST 服务进行通信的最佳实践的演讲？http://www.youtube.com/watch?v=xHXn3Kg2IQE?8m50s（8:50和 11:20）。强烈建议清除最佳实践以及为什么不应在不使用数据库的情况下检索 REST 数据。
In short, consider saving to an sqlite database or a file. If the data is very large you could perhaps consider compressing it before you store it.
简而言之，考虑保存到 sqlite 数据库或文件。如果数据非常大，您或许可以考虑在存储之前对其进行压缩。
回答by Andrey
Maybe this code helps to avoid using StringBuilder and out-of-memory errors:
也许此代码有助于避免使用 StringBuilder 和内存不足错误：
private String convertStreamToString(InputStream is) {
ByteArrayOutputStream oas = new ByteArrayOutputStream();
copyStream(is, oas);
String t = oas.toString();
try {
oas.close();
oas = null;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return t;
}
private void copyStream(InputStream is, OutputStream os)
{
final int buffer_size = 1024;
try
{
byte[] bytes=new byte[buffer_size];
for(;;)
{
int count=is.read(bytes, 0, buffer_size);
if(count==-1)
break;
os.write(bytes, 0, count);
}
}
catch(Exception ex){}
}
回答by Eric Giguere
There are different ways to tackle these kinds of problems. One way would be to use a hash function that doesn't require the entire stream to be in memory, i.e. you feed it a character or block of characters at a time. Another is to reduce the size of the response.
有不同的方法来解决这些类型的问题。一种方法是使用不需要整个流都在内存中的散列函数，即一次向它提供一个字符或字符块。另一个是减少响应的大小。
If you can't do that and you need the entire stream, then I would avoid using readLine() and just call read() on the buffered input stream and append the character you get from read to the string builder. This will reduce the number of strings you are creating and discarding quite dramatically. (A simple optimization to the code above is to take out the newline in the append() call -- you're creating another string unnecessarily there as well.) Also, if you have any idea how long the resulting string will be, you can also set the initial capacity of the string builder upon construction so that you'll know immediately if you'll be out of memory.
如果你不能这样做并且你需要整个流，那么我会避免使用 readLine() 而只是在缓冲的输入流上调用 read() 并将你从 read 获得的字符附加到字符串构建器。这将大大减少您正在创建和丢弃的字符串数量。（对上述代码的一个简单优化是去掉 append() 调用中的换行符——您也在那里创建了另一个不必要的字符串。）此外，如果您知道结果字符串的长度，您还可以在构造时设置字符串生成器的初始容量，以便您立即知道是否会出现内存不足。
Once you get beyond that you have to start splitting the string into blocks that you store on the filesystem.... gets complicated pretty fast.
一旦超出这个范围，您就必须开始将字符串拆分为存储在文件系统上的块……很快就会变得复杂。
回答by sravan
Have you tried the built in method to convert a stream to a string? It's part of the Apache Commons library (org.apache.commons.io.IOUtils).
您是否尝试过将流转换为字符串的内置方法？它是 Apache Commons 库 (org.apache.commons.io.IOUtils) 的一部分。
Then your code would be this one line:
那么你的代码就是这一行：
String total = IOUtils.toString(inputStream);
字符串总计 = IOUtils.toString(inputStream);
The documentation for it can be found here: http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString%28java.io.InputStream%29
它的文档可以在这里找到：http: //commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString%28java.io.InputStream%29
The Apache Commons IO library can be downloaded from here: http://commons.apache.org/io/download_io.cgi
Apache Commons IO 库可以从这里下载：http: //commons.apache.org/io/download_io.cgi
