在终端/bash 脚本中将文件夹拆分为多个子文件夹

Question

提问by Brian C

I have several folders, each with between 15,000 and 40,000 photos. I want each of these to be split into sub folders - each with 2,000 files in them.

我有几个文件夹，每个文件夹都有 15,000 到 40,000 张照片。我希望将其中的每一个都拆分为子文件夹 - 每个子文件夹中有 2,000 个文件。

What is a quick way to do this that will create each folder I need on the go and move all the files?

有什么快速的方法可以在旅途中创建我需要的每个文件夹并移动所有文件？

Currently I can only find how to move the first x items in a folder into a pre-existing directory. In order to use this on a folder with 20,000 items... I would need to create 10 folders manually, and run the command 10 times.

目前我只能找到如何将文件夹中的前 x 个项目移动到预先存在的目录中。为了在包含 20,000 个项目的文件夹上使用它......我需要手动创建 10 个文件夹，并运行该命令 10 次。

ls -1  |  sort -n | head -2000| xargs -i mv "{}" /folder/

I tried putting it in a for-loop, but am having trouble getting it to make folders properly with mkdir. Even after I get around that, I need the program to only create folders for every 20th file (start of a new group). It wants to make a new folder for each file.

我尝试将其放入 for 循环中，但无法使用 mkdir 正确创建文件夹。即使我解决了这个问题，我也需要程序只为每 20 个文件（新组的开始）创建文件夹。它想为每个文件创建一个新文件夹。

So... how can I easily move a large number of files into folders of an arbitrary number of files in each one?

那么...如何轻松地将大量文件移动到每个文件夹中包含任意数量文件的文件夹中？

Any help would be very... well... helpful!

任何帮助都会非常......好吧......很有帮助！

Answer 1

采纳答案by Michael Jaros

This solution can handle names with whitespace and wildcards and can be easily extended to support less straightforward tree structures. It will look for files in all direct subdirectories of the working directory and sort them into new subdirectories of those. New directories will be named 0, 1, etc.:

此解决方案可以处理带有空格和通配符的名称，并且可以轻松扩展以支持不太直接的树结构。它将在工作目录的所有直接子目录中查找文件，并将它们分类到这些目录的新子目录中。新目录将被命名为0，1等：

#!/bin/bash

maxfilesperdir=20

# loop through all top level directories:
while IFS= read -r -d $'for i in `seq 1 20`; do mkdir -p "folder$i"; find . -type f -maxdepth 1 | head -n 2000 | xargs -i mv "{}" "folder$i"; done
' topleveldir
do
        # enter top level subdirectory:
        cd "$topleveldir"

        declare -i filecount=0 # number of moved files per dir
        declare -i dircount=0  # number of subdirs created per top level dir

        # loop through all files in that directory and below
        while IFS= read -r -d $'#!/bin/bash

dir_size=2000
dir_name="folder"
n=$((`find . -maxdepth 1 -type f | wc -l`/$dir_size+1))
for i in `seq 1 $n`;
do
    mkdir -p "$dir_name$i";
    find . -maxdepth 1 -type f | head -n $dir_size | xargs -i mv "{}" "$dir_name$i"
done
' filename
        do
                # whenever file counter is 0, make a new dir:
                if [ "$filecount" -eq 0 ]
                then
                        mkdir "$dircount"
                fi

                # move the file into the current dir:
                mv "$filename" "${dircount}/"
                filecount+=1

                # whenever our file counter reaches its maximum, reset it, and
                # increase dir counter:
                if [ "$filecount" -ge "$maxfilesperdir" ]
                then
                        dircount+=1
                        filecount=0
                fi
        done < <(find -type f -print0)

        # go back to top level:
        cd ..
done < <(find -mindepth 1 -maxdepth 1 -type d -print0)

The find -print0/readcombination with process substitution has been stolen from another question.

与进程替换的find -print0/read组合已从另一个问题中窃取。

It should be noted that simple globbing can handle all kinds of strange directory and file names as well. It is however not easily extensible for multiple levels of directories.

应该注意的是，简单的通配符也可以处理各种奇怪的目录和文件名。然而，对于多个级别的目录，它并不容易扩展。

Answer 2

回答by tmp

Try something like this:

尝试这样的事情：

i=1;while read l;do mkdir $i;mv $l $((i++));done< <(ls|xargs -n2000)

Full script version:

完整脚本版本：

ls|parallel -n2000 mkdir {#}\;mv {} {#}

For dummies:

对于假人：

create a new file: vim split_files.sh
update the dir_sizeand dir_namevalues to match your desires
- note that the dir_namewill have a number appended
navigate into the desired folder: cd my_folder
run the script: sh ../split_files.sh

创建一个新文件： vim split_files.sh
更新dir_size和dir_name值以符合您的需求
- 请注意，dir_name将附加一个数字
导航到所需的文件夹： cd my_folder
运行脚本： sh ../split_files.sh

Answer 3

回答by nisetama

The code below assumes that the filenames do not contain linefeeds, spaces, tabs, single quotes, double quotes, or backslashes, and that filenames do not start with a dash. It also assumes that IFShas not been changed, because it uses while readinstead of while IFS= read, and because variables are not quoted. Add setopt shwordsplitin Zsh.

下面的代码假设文件名不包含换行符、空格、制表符、单引号、双引号或反斜杠，并且文件名不以破折号开头。它还假定IFS未更改，因为它使用了while read代替while IFS= read，并且因为变量没有被引用。添加setopt shwordsplitZsh。

ls|rename --stdin -p 's,^,1+int(($N-1)/2000)."/",e'

The code below assumes that filenames do not contain linefeeds and that they do not start with a dash. -n2000takes 2000 arguments at a time and {#}is the sequence number of the job. Replace {#}with '{=$_=sprintf("%04d",$job->seq())=}'to pad numbers to four digits.

下面的代码假定文件名不包含换行符并且它们不以破折号开头。-n2000一次接受 2000 个参数，{#}是作业的序列号。替换{#}为'{=$_=sprintf("%04d",$job->seq())=}'将数字填充为四位数字。

#!/bin/bash
# outnum generates the name of the output directory
outnum=1
# n is the number of files we have moved
n=0

# Go through all JPG files in the current directory
for f in *.jpg; do
   # Create new output directory if first of new batch of 2000
   if [ $n -eq 0 ]; then
      outdir=folder$outnum
      mkdir $outdir
      ((outnum++))
   fi
   # Move the file to the new subdirectory
   mv "$f" "$outdir"

   # Count how many we have moved to there
   ((n++))

   # Start a new output directory if we have sent 2000
   [ $n -eq 2000 ] && n=0
done

The command below assumes that filenames do not contain linefeeds. It uses the implementation of renameby Aristotle Pagaltzis which is the renameformula in Homebrew, where -pis needed to create directories, where --stdinis needed to get paths from STDIN, and where $Nis the number of the file. In other implementations you can use $.or ++$::iinstead of $N.

下面的命令假定文件名不包含换行符。它使用了renameAristotle Pagaltzis的实现，这是renameHomebrew 中的公式，哪里-p需要创建目录，哪里--stdin需要从 STDIN 获取路径，哪里$N是文件编号。在其他实现中，您可以使用$.或++$::i代替$N。

i=0; for f in *; do d=dir_$(printf %03d $((i/100+1))); mkdir -p $d; mv "$f" $d; let i++; done

Answer 4

回答by Mark Setchell

I would go with something like this:

我会用这样的东西：

#!/bin/bash
# outnum generates the name of the output directory
outnum=1
# n is the number of files we have moved
n=0

if [ "$#" -ne 2 ]; then
    echo Wrong number of args
    echo Usage: bash splitfiles.bash $PWD/directoryoffiles splitsize
    exit 1
fi

# Go through all files in the specified directory
for f in /*; do
   # Create new output directory if first of new batch
   if [ $n -eq 0 ]; then
      outdir=/$outnum
      mkdir $outdir
      ((outnum++))
   fi
   # Move the file to the new subdirectory
   mv "$f" "$outdir"

   # Count how many we have moved to there
   ((n++))

   # Start a new output directory if current new dir is full
   [ $n -eq  ] && n=0
done

Answer 5

回答by Giovanni Benussi

This solutionworked for me on MacOS:

这个解决方案在 MacOS 上对我有用：

#!/bin/bash

dir_size=500
dir_name="folder"
n=$((`find . -maxdepth 1 -type f | wc -l`/$dir_size+1))
for i in `seq 1 $n`;
do
    mkdir -p "$dir_name$i";
    find . -maxdepth 1 -type f | head -n $dir_size | xargs -I '{}' mv {} "$dir_name$i"
done

It creates subfolders of 100 elements each.

它创建每个包含 100 个元素的子文件夹。

Answer 6

回答by ezekiel

This is a tweak of Mark Setchell's

这是 Mark Setchell 的一个调整

Usage: bash splitfiles.bash $PWD/directoryoffiles splitsize

用法：bash splitfiles.bash $PWD/directoryfiles splitsize

It doesn't require the script to be located in the same dir as the files for splitting, it will operate on all files, not just the .jpg and allows you to specify the split size as an argument.

它不需要脚本与要拆分的文件位于同一目录中，它将对所有文件进行操作，而不仅仅是 .jpg，并允许您将拆分大小指定为参数。

NBFiles=$(find . -type f -name *.jpg | wc -l)

Answer 7

回答by u9197735

The answer above is very useful, but there is a very import point in Mac(10.13.6) terminal. Because xargs "-i" argument is not available, I have change the command from above to below.

上面的答案非常有用，但是在 Mac(10.13.6) 终端中有一个非常重要的点。由于 xargs "-i" 参数不可用，我已将命令从上方更改为下方。

ls -1 | sort -n | head -2000| xargs -I '{}' mv {} /folder/

Then, I use the below shell script(reference tmp's answer)

然后，我使用下面的 shell 脚本（参考 tmp 的答案）

NBDIR=$(( $NBFILES / 2000 + 1 ))

Answer 8

回答by jderefinko

You'll certainly have to write a script for that. Hints of things to include in your script:

您当然必须为此编写脚本。要包含在脚本中的提示：

First count the number of files within your source directory

首先计算源目录中的文件数

##代码##

Divide this count by 2000 and add 1, to determine number of directories to create

将此计数除以 2000 并加 1，以确定要创建的目录数

##代码##

Finally loop through your files and move them accross the subdirs. You'll have to use two imbricated loops : one to pick and create the destination directory, the other to move 2000 files in this subdir, then create next subdir and move the next 2000 files to the new one, etc...

最后遍历您的文件并将它们移动到子目录中。您必须使用两个叠层循环：一个选择并创建目标目录，另一个在此子目录中移动 2000 个文件，然后创建下一个子目录并将接下来的 2000 个文件移动到新的，等等......

在终端/bash 脚本中将文件夹拆分为多个子文件夹

提问by Brian C

采纳答案by Michael Jaros

回答by tmp

For dummies:

对于假人：

回答by nisetama

回答by Mark Setchell

回答by Giovanni Benussi

回答by ezekiel

回答by u9197735

回答by jderefinko

相关推荐

最近更新

标签

在终端/bash 脚本中将文件夹拆分为多个子文件夹

提问by Brian C

采纳答案by Michael Jaros

回答by tmp

For dummies:

对于假人：

回答by nisetama

回答by Mark Setchell

回答by Giovanni Benussi

回答by ezekiel

回答by u9197735

回答by jderefinko

相关推荐

匹配后 Bash 打印单词

bash 将命令输出重定向到文件（现有命令）

bash 将文件从驼峰式大小写转换为下划线的 awk/sed 脚本

bash 向 Python 脚本发送消息

相关推荐

最近更新

标签