将文件读取到 byte[] 数组时出现 java.lang.OutOfMemoryError
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/14598857/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
java.lang.OutOfMemoryError while reading file to byte[] array
提问by eWizardII
Is there a cleaner and faster way to do this:
有没有更干净、更快的方法来做到这一点：
BufferedReader inputReader = new BufferedReader(new InputStreamReader(context.openFileInput("data.txt")));
String inputString;
StringBuilder stringBuffer = new StringBuilder();
while ((inputString = inputReader.readLine()) != null) {
stringBuffer.append(inputString + "\n");
}
text = stringBuffer.toString();
byte[] data = text.getBytes();
Basically I'm trying to convert a file into byte[], except if the file is large enough then I run into an outofmemory error. I've been looking around SOfor a solution, I tried to do this here, and it didn't work. Any help would be appreciated.
基本上我正在尝试将文件转换为byte[]，除非文件足够大然后我遇到内存不足错误。我一直在四处寻找SO解决的办法，我想在这里做到这一点，也没有工作。任何帮助，将不胜感激。
采纳答案by Ashwinee K Jha
Few suggestions:
几点建议：
- You don't need to create string builder. You can directly read bytes from the file.
- If you read multiple files, check for those byte[] arrays remaining in memory even when not required.
- Lastly increase the maximum memory for your java process using -Xmx option.
- 您不需要创建字符串生成器。您可以直接从文件中读取字节。
- 如果您读取多个文件，请检查内存中剩余的 byte[] 数组，即使不需要。
- 最后使用 -Xmx 选项增加 java 进程的最大内存。
回答by Audrius Meskauskas
As we know the size of this file, somewhat half of the memory can be saved by allocating the byte array of the given size directly rather than expanding it:
我们知道这个文件的大小，可以通过直接分配给定大小的字节数组而不是扩展它来节省一半的内存：
byte [] data = new byte[ (int) file.length() ];
FileInputStream fin = new FileInputStream(file);
int n = 0;
while ( (n = fin.read(data, n, data.length() - n) ) > 0);
This will avoid allocating unnecessary additional structures. The byte array is only allocated once and has the correct size from beginning. The while loop ensures all data are loaded ( read(byte[], offset, length)may read only part of file but returns the number of bytes read).
这将避免分配不必要的附加结构。字节数组只分配一次并且从一开始就具有正确的大小。while 循环确保加载所有数据（read(byte[], offset, length)可能只读取文件的一部分，但返回读取的字节数）。
Clarification:When the StringBuilder runs out, it allocates a new buffer that is the two times larger than the initial buffer. At this moment, we are using about twice the amount of memory that would be minimally required. In the most degenerate case (one last byte does not fit into some already big buffer), near three times the minimal amount of RAM may be required.
说明：当 StringBuilder 用完时，它会分配一个比初始缓冲区大两倍的新缓冲区。目前，我们使用的内存量大约是最低要求的两倍。在最简陋的情况下（最后一个字节不适合一些已经很大的缓冲区），可能需要的 RAM 量几乎是最小量的三倍。
回答by Andremoniy
If you haven't enough memory to store there whole file, you can try rethink your algorithm to process file data while reading it, without constructing large byte[]array data.
如果您没有足够的内存来存储整个文件，您可以尝试重新思考算法以在读取文件数据时处理文件数据，而无需构建大型byte[]数组数据。
If you have already tried increase javamemory by playing with -Xmxparameter, then there isn't any solution, which will allow you store data in memory, which can not be located there due to its large size.
如果你已经尝试过java通过玩-Xmx参数来增加内存，那么没有任何解决方案，它可以让你将数据存储在内存中，因为内存大而无法定位。
回答by user207421
The 'cleaner and faster way' is not to do it at all. It doesn't scale. Process the file a piece at a time.
“更干净、更快捷的方式”是根本不做。它没有规模。一次处理一个文件。
回答by Peter Lawrey
You are copying bytes into char (which use twice the space) and back into bytes again.
您正在将字节复制到 char（使用两倍的空间）并再次复制回字节。
InputStream in = context.openFileInput("data.txt");
ByteArrayOutputStream bais = new ByteArrayOutputStream();
byte[] bytes = new byte[8192];
for(int len; (lne = in.read(bytes) > 0;)
bais.write(bytes, 0, len);
in.close();
return bais.toByteArray();
This will half your memory requirement but it can still mean you run out of memory. In this case you have to either
这将使您的内存需求减半，但仍可能意味着您的内存不足。在这种情况下，您必须要么
- increase your maximum heap size
- process the file progressively instead of all at once
- use memory mapped files which allows you to "load" a file without using much heap.
- 增加最大堆大小
- 逐步处理文件而不是一次处理所有文件
- 使用内存映射文件，它允许您在不使用太多堆的情况下“加载”文件。
回答by lumiera
This is similar to File to byte[] in Java
You're currently reading in bytes, converting them to characters, and then trying to turn them back into bytes. From the InputStreamReader class in the Java API:
您当前正在读取字节，将它们转换为字符，然后尝试将它们转换回字节。从 Java API 中的 InputStreamReader 类：
An InputStreamReader is a bridge from byte streams to character streams: It reads bytes and decodes them into characters..
InputStreamReader 是从字节流到字符流的桥梁：它读取字节并将它们解码为字符。
It would be way more efficient to just read in bytes.
仅以字节为单位读取会更有效。
One way would be to use a ByteArrayInputStreamdirectly on context.openFileInput(), or the Jakarta Commons IOUtils.toByteArray(InputStream), or if you're using JDK7 you can use Files.readAllBytes(Path).
一种方法是ByteArrayInputStream直接在context.openFileInput()或 Jakarta Commons 上使用IOUtils.toByteArray(InputStream)，或者如果您使用的是 JDK7，则可以使用Files.readAllBytes(Path).
回答by JayTee
This solution will test the free memory before loading...
此解决方案将在加载之前测试可用内存...
File test = new File("c:/tmp/example.txt");
long freeMemory = Runtime.getRuntime().freeMemory();
if(test.length()<freeMemory) {
byte[] bytes = new byte[(int) test.length()];
FileChannel fc = new FileInputStream(test).getChannel();
MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_ONLY, 0, (int) fc.size());
while(mbb.hasRemaining()) {
mbb.get(bytes);
}
fc.close();
}
