Linux 如何在不读取整个文件的情况下从大文件的末尾删除 X 个字节?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/7392444/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-05 06:09:54  来源:igfitidea点击:

How to remove X bytes from the end of a large file without reading the whole file?

linuxshell

提问by andyortlieb

In Linux, I have a rather large file with some extraneous information tacked on to the end of it. Let's say for example I know there are 314 bytes of extraneous data at the end of a 1.6GB file.

在 Linux 中,我有一个相当大的文件,在它的末尾附加了一些无关的信息。例如,假设我知道 1.6GB 文件的末尾有 314 字节的无关数据。

Of course it is very easy and efficient to add more data to the end of a file, but what can I do to remove it without having to copy the first portion of that file into another (or overwrite said file)?

当然,将更多数据添加到文件末尾是非常容易和高效的,但是我可以做些什么来删除它而不必将该文件的第一部分复制到另一个(或覆盖所述文件)?

Edit

编辑

I'm seeing some good advice on doing this in C. I was hoping to script it from the commandline, but failing that I would be more inclined to doing it in python than C.

我看到一些关于在 C 中执行此操作的好建议。我希望从命令行编写它的脚本,但如果失败,我会比 C 更倾向于在 python 中执行此操作。

I see that python has a truncate method on its file object but it seems to be demolishing my file no matter how i use it--I should be able to figure this one out, but of course answers are more than welcome still.

我看到 python 在其文件对象上有一个 truncate 方法,但无论我如何使用它,它似乎都在破坏我的文件——我应该能够弄清楚这一点,但当然答案仍然非常受欢迎。

采纳答案by KevinDTimm

use the function truncate

使用函数 truncate

http://linux.die.net/man/2/truncate

http://linux.die.net/man/2/truncate

int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length); 

truncate takes the file name
ftruncate takes an open file descriptor

truncate 取文件名
ftruncate 取一个打开的文件描述符

both of these set the file length to lengthso it either truncates or elongates (in the latter case, the rest of the file will be filled with NULL/ZERO)

这两个都将文件长度设置length为截断或拉长(在后一种情况下,文件的其余部分将填充为空/零)

[edit]
truncate(linux shell command) will work also

[编辑]
truncate(linux shell 命令)也可以使用

**SYNTAX**

truncate -s integer <filename>  
**OPTIONS**

-s number specify the new file length. If the new length is smaller than the current filelength data is lost. If the new length is greater the file is padded with 0. You can specify a magnitude character to ease large numbers:
b or B size is bytes.
k size is 1000 bytes.
K size is 1024 bytes.
m size is 10^6 bytes.
M size is 1024^2 bytes.
g size is 10^9 bytes.
G size is 1024^3 bytes.


**EXAMPLES**

To shrink a file to 10 bytes:

truncate -s 10 /tmp/foo

To enlarge or shrink a file to 345 Megabytes:

truncate -s 345M /tmp/foo

[/edit]

[/编辑]

回答by Tom Zych

In C on a POSIX-compliant system (or more generally, most Unix-like systems), you can use the truncateand ftruncatefunctions.

在符合 POSIX 的系统(或更一般地说,大多数类 Unix 系统)上的 C 中,您可以使用truncateftruncate函数。

回答by sherrellbc

Although there were plenty of references to the truncatefunction in this thread, no one really answered the OP's question about reducing a file by a fixed amount from a scripting environment. Kevin's answer used truncateto resize the target file to a fixed amount, but of course the correctness of this solution requires the user to first know the size of the target file minus the extraneous data at the end. So, we have:

尽管truncate该线程中有大量对该函数的引用,但没有人真正回答 OP 关于从脚本环境中将文件减少固定数量的问题。Kevin 的答案用于truncate将目标文件大小调整为固定数量,但当然这个解决方案的正确性要求用户首先知道目标文件的大小减去最后的无关数据。所以,我们有:

   -s, --size=SIZE
          set or adjust the file size by SIZE bytes
   -s, --size=SIZE
          set or adjust the file size by SIZE bytes

Truncateactually supports removing data from the end of a target file directly by prefixing SIZE with a -character.

Truncate实际上支持通过在 SIZE 前面加上-字符来直接从目标文件的末尾删除数据。

For example, to resize a file by 314 bytes you can do:

例如,要将文件大小调整为 314 字节,您可以执行以下操作:

truncate --size=-314 target_file.bin

truncate --size=-314 target_file.bin

回答by Hannah Zhang

Using "truncate" is best way, I just post some examples:

使用“截断”是最好的方法,我只是发布了一些例子:

  1. I have a file "PNav-h.db", it has 50176 bytes.

    -rw-r--r--  1 user user 50176 Mar  8 23:43 PNav-h.db
     $truncate -s 1000 PNav-h.db
    

    it set the file size to 1000 byptes

     -rw-r--r--  1 user user  1000 Mar  9 00:02 PNav-h.db
    
  2. For your case, use $truncate --size=xxx xxfilename, using -<size number>to reduce the file size

    $truncate --size=-300 PNav-h.db
    -rw-r--r--  1 user user   700 Mar  9 00:07 PNav-h.db
    

    final file size = 1000-300=700

  3. using +<size number>to increase the file size

    $truncate --size=+500 PNav-h.db
    -rw-r--r--  1 user user  1200 Mar  9 00:09 PNav-h.db
    

    final file size = 700 + 500 = 1200

  4. if there is no -or +, it is to set the file size.

    $truncate --size=60000 PNav-h.db
    -rw-r--r--  1 user user 60000 Mar  9 00:12 PNav-h.db 
    

    final file size set to 60000

  1. 我有一个文件“PNav-h.db”,它有 50176 个字节。

    -rw-r--r--  1 user user 50176 Mar  8 23:43 PNav-h.db
     $truncate -s 1000 PNav-h.db
    

    它将文件大小设置为 1000 字节

     -rw-r--r--  1 user user  1000 Mar  9 00:02 PNav-h.db
    
  2. 对于您的情况,请使用$truncate --size=xxx xxfilename, using-<size number>来减小文件大小

    $truncate --size=-300 PNav-h.db
    -rw-r--r--  1 user user   700 Mar  9 00:07 PNav-h.db
    

    最终文件大小 = 1000-300=700

  3. 通过+<size number>增加文件大小

    $truncate --size=+500 PNav-h.db
    -rw-r--r--  1 user user  1200 Mar  9 00:09 PNav-h.db
    

    最终文件大小 = 700 + 500 = 1200

  4. 如果没有-+,则是设置文件大小。

    $truncate --size=60000 PNav-h.db
    -rw-r--r--  1 user user 60000 Mar  9 00:12 PNav-h.db 
    

    最终文件大小设置为 60000