bash 替代scp,通过打开并行连接在linux机器之间传输文件

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/2970460/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-17 22:10:04  来源:igfitidea点击:

Alternative to scp, transferring files between linux machines by opening parallel connections

linuxbashunixnetworkingshell

提问by Boolean

Is there an alternative to scp, to transfer a large file from one machine to another machine by opening parallel connections and also able to pause and resume the download.

是否有 scp 的替代方法,通过打开并行连接将大文件从一台机器传输到另一台机器,还可以暂停和恢复下载。

Please don't transfer this to severfault.com. I am not a system administrator. I am a developer trying to transfer past database dumps between backup hosts and servers.

请不要将此转移到 severfault.com。我不是系统管理员。我是一名开发人员,试图在备份主机和服务器之间传输过去的数据库转储。

Thank you

谢谢

回答by MikeK

You could try using split(1) to break the file apart and then scp the pieces in parallel. The file could then be combined into a single file on the destination machine with 'cat'.

您可以尝试使用 split(1) 将文件分开,然后并行 scp 各个部分。然后可以使用“cat”将该文件组合成目标计算机上的单个文件。

# on local host
split -b 1M large.file large.file. # split into 1MiB chunks
for f in large.file.*; do scp $f remote_host: & done

# on remote host
cat large.file.* > large.file

回答by danw

Similar to Mike K's answer, check out https://code.google.com/p/scp-tsunami/- it handles splitting the file, starting several scp processes to copy the parts and then joins them again...it can also copy to multiple hosts...

与 Mike K 的回答类似,请查看https://code.google.com/p/scp-tsunami/- 它处理拆分文件,启动多个 scp 进程以复制部分然后再次加入它们......它也可以复制到多个主机...

 ./scpTsunami.py -v -s -t 9 -b 10m -u dan bigfile.tar.gz /tmp -l remote.host

That splits the file into 10MB chunks and copies them using 9 scp processes...

这将文件分成 10MB 块并使用 9 个 scp 进程复制它们......

回答by Paused until further notice.

Take a look at rsyncto see if it will meet your needs.

看看rsync看看它是否能满足您的需求。

The correct placement of questions is not based on your role, but on the type of question. Since this one is not strictly programming related it is likely that it will be migrated.

问题的正确放置不是基于您的角色,而是基于问题的类型。由于这与严格的编程无关,因此它很可能会被迁移。

回答by Anonymous

The program you are after is lftp. It supports sftp and parallel transfers using its pget command. It is available under Ubuntu (sudo apt-get install lftp) and you can read a review of it here:

你所追求的程序是lftp。它使用 pget 命令支持 sftp 和并行传输。它在 Ubuntu 下可用(sudo apt-get install lftp),你可以在这里阅读它的评论:

http://www.cyberciti.biz/tips/linux-unix-download-accelerator.html

http://www.cyberciti.biz/tips/linux-unix-download-accelerator.html