java 用java编写时限制文件大小
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/39092861/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Limit file size while writing in java
提问by hacker
I need to limit the file size to 1 GB while writing preferably using BufferedWriter.
我需要将文件大小限制为 1 GB，同时最好使用BufferedWriter.
Is it possible using BufferedWriteror I have to use other libraries ?
是否可以使用BufferedWriter或我必须使用其他库？
like
喜欢
try (BufferedWriter writer = Files.newBufferedWriter(path)) {
//...
writer.write(lines.stream());
}
回答by Andreas
You can always write your own OutputStreamto limit the number of byteswritten.
您始终可以自己编写OutputStream以限制写入的字节数。
The following assumes you want to throw exception if size is exceeded.
以下假设您想在超出大小时抛出异常。
public final class LimitedOutputStream extends FilterOutputStream {
private final long maxBytes;
private long bytesWritten;
public LimitedOutputStream(OutputStream out, long maxBytes) {
super(out);
this.maxBytes = maxBytes;
}
@Override
public void write(int b) throws IOException {
ensureCapacity(1);
super.write(b);
}
@Override
public void write(byte[] b) throws IOException {
ensureCapacity(b.length);
super.write(b);
}
@Override
public void write(byte[] b, int off, int len) throws IOException {
ensureCapacity(len);
super.write(b, off, len);
}
private void ensureCapacity(int len) throws IOException {
long newBytesWritten = this.bytesWritten + len;
if (newBytesWritten > this.maxBytes)
throw new IOException("File size exceeded: " + newBytesWritten + " > " + this.maxBytes);
this.bytesWritten = newBytesWritten;
}
}
You will of course now have to set up the Writer/OutputStreamchain manually.
当然，您现在必须手动设置Writer/OutputStream链。
final long SIZE_1GB = 1073741824L;
try (BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
new LimitedOutputStream(Files.newOutputStream(path), SIZE_1GB),
StandardCharsets.UTF_8))) {
//
}
回答by CoderCroc
Exact bytes to 1 GB is very difficult in cases where you are writing lines. Each line may contain unknown number of bytes in it. I am assuming you want to write data line by line in file.
在您写行的情况下，精确到 1 GB 的字节是非常困难的。每行可能包含未知数量的字节。我假设您想在文件中逐行写入数据。
However, you can check how many bytes does line has before writing it to the file and another approach is to check file size after writing each line.
但是，您可以在将行写入文件之前检查行有多少字节，另一种方法是在写入每一行后检查文件大小。
Following basic example writes one same line each time. Here This is just a test !text takes 21 bytes on file in UTF-8 encoding. Ultimately after 49 writes it reaches to 1029 Bytes and stops writing.
以下基本示例每次写入相同的一行。这里这只是一个测试！text 在 UTF-8 编码的文件中占用 21 个字节。最终在 49 次写入后达到 1029 字节并停止写入。
public class Test {
private static final int ONE_KB = 1024;
public static void main(String[] args) {
File file = new File("D:/test.txt");
try (BufferedWriter writer = Files.newBufferedWriter(file.toPath())) {
while (file.length() < ONE_KB) {
writer.write("This is just a test !");
writer.flush();
}
System.out.println("1 KB Data is written to the file.!");
} catch (IOException e) {
e.printStackTrace();
}
}
}
As you can see we have already written out of the limit of 1KB as above program writes 1029 Bytes and not less than 1024 Bytes.
如您所见，我们已经超出了 1KB 的限制，如上程序写入 1029 字节且不少于 1024 字节。
Second approach is checking the bytes according to specific encoding before writing it to file.
第二种方法是在将字节写入文件之前根据特定编码检查字节。
public class Test {
private static final int ONE_KB = 1024;
public static void main(String[] args) throws UnsupportedEncodingException {
File file = new File("D:/test.txt");
String data = "This is just a test !";
int dataLength = data.getBytes("UTF-8").length;
try (BufferedWriter writer = Files.newBufferedWriter(file.toPath())) {
while (file.length() + dataLength < ONE_KB) {
writer.write(data);
writer.flush();
}
System.out.println("1 KB Data written to the file.!");
} catch (IOException e) {
e.printStackTrace();
}
}
}
In this approach we check length of bytes prior to writing it to the file. So, it will write 1008 Bytes and it will stop writing.
在这种方法中，我们在将其写入文件之前检查字节长度。因此，它将写入 1008 字节，然后停止写入。
Problems with both the approaches,
两种方法都有问题，
- Write and Check : You may end up with some extra bytes and file size may cross the limit
- Check and Write : You may have less bytes than the limit if next line has lot of data in it. You should be careful about the encoding.
- 写入和检查：您最终可能会得到一些额外的字节，并且文件大小可能会超过限制
- 检查和写入：如果下一行中有大量数据，则字节数可能少于限制。您应该注意编码。
However, there are other ways to do this validations with some third party library like apache ioand I find it more cumbersome then conventional java ways.
但是，还有其他方法可以使用某些第三方库（如apache io）进行此验证，我发现它比传统的 java 方法更麻烦。
回答by Joop Eggen
int maxSize = 1_000_000_000;
Charset charset = StandardCharsets.UTF_F);
int size = 0;
int lineCount = 0;
while (lineCount < lines.length) {
long size2 = size + (lines[lineCount] + "\r\n").getBytes(charset).length;
if (size2 > maxSize) {
break;
}
size = size2;
++lineCount;
}
List<String> linesToWrite = lines.substring(0, lineCount);
Path path = Paths.get("D:/test.txt");
Files.write(path, linesToWrite , charset);
Or a bit faster while decoding only once:
或者只解码一次时更快一点：
int lineCount = 0;
try (FileChannel channel = new RandomAccessFile("D:/test.txt", "w").getChannel()) {
ByteBuffer buf = channel.map(FileChannel.MapMode.WRITE, 0, maxSize);
lineCount = lines.length;
for (int i = 0; i < lines.length; i++) {
bytes[] line = (lines.get(i) + "\r\n").getBytes(charset);
if (line.length > buffer.remaining()) {
lineCount = i;
break;
}
buffer.put(line);
}
}
回答by Sanjeev Dhiman
IIUC, there are various ways to do it.
IIUC，有多种方法可以做到。
- Keep writing data in chucks and flushing it and keep checking the file size after every flush.
- Use log4j (or some logging framework) which can let us rollover to new file after certain size or time or some other trigger point.
- While BufferedReader is great, there are some new APIs in java which could make it faster. Fastest way to write huge data in text file Java
- 继续在卡盘中写入数据并刷新它，并在每次刷新后继续检查文件大小。
- 使用 log4j（或一些日志框架），它可以让我们在特定大小或时间或其他触发点后滚动到新文件。
- 虽然 BufferedReader 很棒，但 Java 中有一些新的 API 可以使它更快。在文本文件Java中写入大量数据的最快方法
