Linux 使用 tar、gz、zip 或 bzip2 拆分文件

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/1120095/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-03 17:31:30  来源:igfitidea点击:

Split files using tar, gz, zip, or bzip2

linuxbashfile-iocompression

提问by Aka

I need to compress a large file of about 17-20 GB. I need to split it into several files of around 1GB per file.

我需要压缩一个大约 17-20 GB 的大文件。我需要将它分成几个文件,每个文件大约 1GB。

I searched for a solution via Google and found ways using splitand catcommands. But they did not work for large files at all. Also, they won't work in Windows; I need to extract it on a Windows machine.

我通过谷歌搜索了一个解决方案,并找到了使用splitcat命令的方法。但它们根本不适用于大文件。此外,它们在 Windows 中不起作用;我需要在 Windows 机器上提取它。

回答by Adrian Panasiuk

Tested code, initially creates a single archive file, then splits it:

经过测试的代码,最初创建一个存档文件,然后将其拆分:

 gzip -c file.orig > file.gz
 CHUNKSIZE=1073741824
 PARTCNT=$[$(stat -c%s file.gz) / $CHUNKSIZE]

 # the remainder is taken care of, for example for
 # 1 GiB + 1 bytes PARTCNT is 1 and seq 0 $PARTCNT covers
 # all of file
 for n in `seq 0 $PARTCNT`
 do
       dd if=file.gz of=part.$n bs=$CHUNKSIZE skip=$n count=1
 done

This variant omits creating a single archive file and goes straight to creating parts:

此变体省略了创建单个存档文件并直接创建零件:

gzip -c file.orig |
    ( CHUNKSIZE=1073741824;
        i=0;
        while true; do
            i=$[i+1];
            head -c "$CHUNKSIZE" > "part.$i";
            [ "$CHUNKSIZE" -eq $(stat -c%s "part.$i") ] || break;
        done; )

In this variant, if the archive's file size is divisible by $CHUNKSIZE, then the last partial file will have file size 0 bytes.

在此变体中,如果存档的文件大小可被 整除$CHUNKSIZE,则最后一部分文件的文件大小将为 0 字节。

回答by Joshua

If you are splitting from Linux, you can still reassemble in Windows.

如果您要从 Linux 拆分,您仍然可以在 Windows 中重新组装。

copy /b file1 + file2 + file3 + file4 filetogether

回答by Tim Hoolihan

use tar to split into multiple archives

使用 tar拆分成多个存档

there are plenty of programs that will work with tar files on windows, including cygwin.

有很多程序可以在 Windows 上处理 tar 文件,包括 cygwin。

回答by matpie

You can use the splitcommand with the -boption:

您可以使用split带有以下-b选项的命令:

split -b 1024m file.tar.gz

It can be reassembled on a Windows machine using @Joshua's answer.

可以使用 @ Joshua的答案在 Windows 机器上重新组装它。

copy /b file1 + file2 + file3 + file4 filetogether


Edit: As @Charlie stated in the comment below, you might want to set a prefix explicitly because it will use xotherwise, which can be confusing.

编辑:正如@Charlie 在下面的评论中所述,您可能希望显式设置前缀,因为它会以x其他方式使用,这可能会造成混淆。

split -b 1024m "file.tar.gz" "file.tar.gz.part-"

// Creates files: file.tar.gz.part-aa, file.tar.gz.part-ab, file.tar.gz.part-ac, ...


Edit: Editing the post because question is closed and the most effective solution is very close to the content of this answer:

编辑:编辑帖子,因为问题已关闭,最有效的解决方案与此答案的内容非常接近:

# create archives
$ tar cz my_large_file_1 my_large_file_2 | split -b 1024MiB - myfiles_split.tgz_
# uncompress
$ cat myfiles_split.tgz_* | tar xz

This solution avoids the need to use an intermediate large file when (de)compressing. Use the tar -C option to use a different directory for the resulting files. btw if the archive consists from only a single file, tar could be avoided and only gzip used:

此解决方案避免了在(解)压缩时使用中间大文件的需要。使用 tar -C 选项为结果文件使用不同的目录。顺便说一句,如果存档仅由一个文件组成,则可以避免使用 tar 并且只使用 gzip:

# create archives
$ gzip -c my_large_file | split -b 1024MiB - myfile_split.gz_
# uncompress
$ cat myfile_split.gz_* | gunzip -c > my_large_file

For windows you can download ported versions of the same commands or use cygwin.

对于 Windows,您可以下载相同命令的移植版本或使用 cygwin。