bash 保证文件夹的原子移动

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/9839823/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 01:50:48  来源:igfitidea点击:

Guaranteed atomic move of folder

linuxbash

提问by natli

I have a script that runs every minute at XX:00. The script loops over all subfolders in a given directory and performs operations on the files inside;

我有一个每分钟在 XX:00 运行的脚本。该脚本遍历给定目录中的所有子文件夹并对其中的文件执行操作;

folder=/path/to/directory #Starting directory
someerror=0 #Did we have an error?

#CD to directory. Does it even exist?
cd $folder
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success changing directory to $folder && mainfolderexist=1
[ $RETVAL -ne 0 ] && echo Failure changing directory to $folder && mainfolderexist=0

if [ $mainfolderexist -eq 1 ]; then
    shopt -s nullglob
    for dir in $folder/*/
    do
    thedirname=`basename $dir` #Get directory name
    #cd to sub dir. Does it even exist?
    cd $dir
    RETVAL=$?
    [ $RETVAL -eq 0 ] && echo Success changing directory to $dir && subfolderexist=1
    [ $RETVAL -ne 0 ] && echo Failure changing directory to $dir && subfolderexist=0
    if [ $subfolderexist -eq 1 ]; then
        #perform some operation on all files in this directory
        someApp -someArgs --name=$thedirname *
    else #sub folder doesn't exist
        someerror=1
        break
    fi

    #next folder
    done
else #main folder doesn't exist
    someerror=1
fi

#REPEAT (only if no errors occured)
if [ $someerror -eq 0 ]; then
at now + 1 minutes << END
/bin/bash "##代码##" "$@"
END
fi

The way I use this, is I upload directories to the server using SFTP, to a folder like /home/incoming, and once the directory is fully uploaded I will move it to /path/to/directory. Now this is the part I am worried about.

我用这个问题的方法,是我上传的目录使用SFTP服务器,像一个文件夹/home/incoming,一旦该目录被完全上传,我将它移到/path/to/directory。现在这是我担心的部分。

So far I've been making sure to only move directories between XX:XX:02 and XX:XX:50, but is this even neccesary? I would like to automate the upload+move process without taking system time into account so I am wondering;

到目前为止,我一直确保只在 XX:XX:02 和 XX:XX:50 之间移动目录,但这甚至是必要的吗?我想在不考虑系统时间的情况下自动执行上传+移动过程,所以我想知道;

  1. What if a directory is in the process of getting moved (mv "somedir" "/path/to/directory/somedir") at XX:00 and the script runs, looping over all directories?
  2. What if the system loses power during the mv command? If the directory will end up half moved or something similar, I will have to write a script verifying this before executing the above script.
  1. 如果一个目录正在mv "somedir" "/path/to/directory/somedir"XX:00移动 ( ) 并且脚本运行,循环遍历所有目录怎么办?
  2. 如果系统在 mv 命令期间断电怎么办?如果目录最终移动了一半或类似的东西,我将不得不在执行上述脚本之前编写一个脚本来验证这一点。

回答by larsks

If your source and destination paths are on the same filesystem, then mvis an atomic operation. Since it does not actually involve copying or otherwise relocating files, your directories will never end up in a "half-moved" state.

如果源路径和目标路径在同一个文件系统上,则mv是原子操作。由于它实际上并不涉及复制或以其他方式重新定位文件,因此您的目录永远不会处于“半移动”状态。

If, on the other hand, your source and destination paths are on differentfilesystems, then mvis actually a copy followed by a delete over the entire tree, which can take a substantial amount of time and, if interrupted, will leave things in a half-completed state.

另一方面,如果您的源路径和目标路径位于不同的文件系统上,那么mv实际上是先复制后删除整个树,这可能会花费大量时间,并且如果中断,则会将事情拖到一半- 完成状态。

回答by Jeff Foster

I believe that mvis an atomic operation (moving a directory just renames the directory, it doesn't move any files on disk). mvmakes the sys-call renamewhich according to thisis atomic (subject to some constraints).

我相信这mv是一个原子操作(移动目录只是重命名目录,它不会移动磁盘上的任何文件)。 mv使系统调用rename根据this是原子的(受某些限制)。

If newpath already exists it will be atomically replaced (subject to a few conditions - see ERRORS below), so that there is no point at which another process attempting to access newpath will find it missing.

如果 newpath 已经存在,它将被自动替换(取决于一些条件 - 请参见下面的错误),因此在尝试访问 newpath 的另一个进程不会发现它丢失的时候。

回答by user1695170

Remember the movement is not atomic even if you are in the same disc and partition when either one (or both) is using encryption as a software. Eg: 2 users in the same machine have directories in the /home. If one of then uses an encrypted home directory, the move will not be atomic because it will need to encrypt/decrypt or decrypt/encrypt or encrypt or decrypt the files before make the move

请记住,即使您在同一个磁盘和分区中,当其中一个(或两个)将加密用作软件时,移动也不是原子的。例如:同一台机器上的 2 个用户在 /home 中有目录。如果其中之一使用加密的主目录,则移动将不是原子的,因为它需要在移动之前加密/解密或解密/加密或加密或解密文件