bash 对目录中的所有文件执行命令

Execute command on all files in a directory

提问by themaestro

Could somebody please provide the code to do the following: Assume there is a directory of files, all of which need to be run through a program. The program outputs the results to standard out. I need a script that will go into a directory, execute the command on each file, and concat the output into one big output file.

有人可以提供代码来执行以下操作：假设有一个文件目录，所有这些文件都需要通过程序运行。程序将结果输出到标准输出。我需要一个脚本，它会进入一个目录，对每个文件执行命令，并将输出连接到一个大的输出文件中。

For instance, to run the command on 1 file:

例如，要在 1 个文件上运行命令：

$ cmd [option] [filename] > results.out

回答by Andrew Logvinov

The following bash code will pass $file to command where $file will represent every file in /dir

以下 bash 代码将 $file 传递给命令，其中 $file 将代表 /dir 中的每个文件

for file in /dir/*
do
  cmd [option] "$file" >> results.out
done

Example

例子

el@defiant ~/foo $ touch foo.txt bar.txt baz.txt
el@defiant ~/foo $ for i in *.txt; do echo "hello $i"; done
hello bar.txt
hello baz.txt
hello foo.txt

回答by Jim Lewis

How about this:

这个怎么样：

find /some/directory -maxdepth 1 -type f -exec cmd option {} \; > results.out

• -maxdepth 1argument prevents find from recursively descending into any subdirectories. (If you want such nested directories to get processed, you can omit this.)
• -type -fspecifies that only plain files will be processed.
• -exec cmd option {}tells it to run cmdwith the specified optionfor each file found, with the filename substituted for {}
• \;denotes the end of the command.
• Finally, the output from all the individual cmdexecutions is redirected to results.out

• -maxdepth 1参数防止 find 递归下降到任何子目录。（如果您希望处理此类嵌套目录，则可以省略此项。）
• -type -f指定仅处理纯文件。
• -exec cmd option {}告诉它cmd使用option为找到的每个文件指定的运行，文件名替换为{}
• \;表示命令的结束。
• 最后，所有单独cmd执行的输出被重定向到 results.out

However, if you care about the order in which the files are processed, you might be better off writing a loop. I think findprocesses the files in inode order (though I could be wrong about that), which may not be what you want.

但是，如果您关心处理文件的顺序，最好编写一个循环。我认为find按 inode 顺序处理文件（尽管我可能错了），这可能不是您想要的。

回答by robgraves

I'm doing this on my raspberry pi from the command line by running:

我正在从命令行通过运行在我的树莓派上执行此操作：

for i in *;do omxplayer "$i";done

回答by Inian

The accepted/high-voted answers are great, but they are lacking a few nitty-gritty details. This post covers the cases on how to better handle when the shell path-name expansion (glob) fails, when filenames contain embedded newlines/dash symbols and moving the command output re-direction out of the for-loop when writing the results to a file.

接受/高投票的答案很好，但它们缺乏一些细节。这篇文章介绍了如何更好地处理 shell 路径名扩展 (glob) 失败、文件名包含嵌入的换行符/破折号以及将命令输出重定向移出 for 循环时将结果写入到文件。

When running the shell glob expansion using *there is a possibility for the expansion to fail if there are nofiles present in the directory and an un-expanded glob string will be passed to the command to be run, which could have undesirable results. The bashshell provides an extended shell option for this using nullglob. So the loop basically becomes as follows inside the directory containing your files

使用 shell glob 扩展运行时，*如果目录中不存在文件，则扩展可能会失败，并且未扩展的 glob 字符串将传递给要运行的命令，这可能会产生不良结果。所述bash外壳提供了用于此使用扩展壳选项nullglob。所以循环在包含文件的目录中基本上变成如下

 shopt -s nullglob

 for file in ./*; do
     cmdToRun [option] -- "$file"
 done

This lets you safely exit the for loop when the expression ./*doesn't return any files (if the directory is empty)

这使您可以在表达式./*不返回任何文件时安全地退出 for 循环（如果目录为空）

or in a POSIX compliant way (nullglobis bashspecific)

或以符合 POSIX 的方式（nullglob是bash特定的）

 for file in ./*; do
     [ -f "$file" ] || continue
     cmdToRun [option] -- "$file"
 done

This lets you go inside the loop when the expression fails for once and the condition [ -f "$file" ]check if the un-expanded string ./*is a valid filename in that directory, which wouldn't be. So on this condition failure, using continuewe resume back to the forloop which won't run subsequently.

这使您可以在表达式失败一次时进入循环，并且条件[ -f "$file" ]检查未扩展的字符串./*是否是该目录中的有效文件名，而事实并非如此。因此，在这种情况下失败时，使用continue我们恢复到for随后不会运行的循环。

Also note the usage of --just before passing the file name argument. This is needed because as noted previously, the shell filenames can contain dashes anywhere in the filename. Some of the shell commands interpret that and treat them as a command option when the name are notquoted properly and executes the command thinking if the flag is provided.

还要注意--在传递文件名参数之前的用法。这是必需的，因为如前所述，shell 文件名可以在文件名的任何位置包含破折号。当名称未正确引用并执行命令时，某些 shell 命令将其解释并视为命令选项，并认为是否提供了标志。

The --signals the end of command line options in that case which means, the command shouldn't parse any strings beyond this point as command flags but only as filenames.

--在这种情况下，该信号表示命令行选项的结束，这意味着该命令不应将超出此点的任何字符串解析为命令标志，而只能解析为文件名。

---

---

Double-quoting the filenames properly solves the cases when the names contain glob characters or white-spaces. But *nix filenames can also contain newlines in them. So we de-limit filenames with the only character that cannot be part of a valid filename - the null byte (\0). Since bashinternally uses Cstyle strings in which the null bytes are used to indicate the end of string, it is the right candidate for this.

双引号文件名正确解决了名称包含全局字符或空格的情况。但是 *nix 文件名中也可以包含换行符。因此，我们使用唯一不能作为有效文件名一部分的字符（空字节 ( \0)）来限制文件名。由于bash内部使用C样式字符串，其中空字节用于指示字符串的结尾，因此它是正确的候选者。

So using the printfoption of shell to delimit files with this NULL byte using the -doption of readcommand, we can do below

所以使用printfshell的-d选项使用readcommand的选项来用这个NULL字节分隔文件，我们可以在下面做

( shopt -s nullglob; printf '%s
while IFS= read -r -d '' file; do
    cmdToRun [option] -- "$file"
done < <(find -maxdepth 1 -type f -print0)
' ./* ) | while read -rd '' file; do
    cmdToRun [option] -- "$file"
done

The nullgloband the printfare wrapped around (..)which means they are basically run in a sub-shell (child shell), because to avoid the nullgloboption to reflect on the parent shell, once the command exits. The -d ''option of readcommand is notPOSIX compliant, so needs a bashshell for this to be done. Using findcommand this can be done as

在nullglob与printf被缠(..)他们基本上在一个子shell（子shell）运行，这意味着，避免因为nullglob反思父shell，一旦命令退出选项。该-d ''的选项read命令是不符合POSIX标准，所以需要一个bash壳为此做。使用find命令可以这样做

find . -maxdepth 1 -type f -exec printf '%s
( shopt -s nullglob; printf '%s
while IFS= read -r -d '' file; do
    cmdToRun [option] -- "$file"
done < <(find -maxdepth 1 -type f -print0) > results.out
' ./* ) | while read -rd '' file; do
    cmdToRun [option] -- "$file"
done > results.out
' {} \; | xargs -0 cmdToRun [option] --

For findimplementations that don't support -print0(other than the GNU and the FreeBSD implementations), this can be emulated using printf

对于find不支持的实现-print0（除了 GNU 和 FreeBSD 实现），这可以使用printf

find directory/ | xargs  Command 

---

---

Another important fix is to move the re-direction out of the for-loop to reduce a high number of file I/O. When used inside the loop, the shell has to execute system-calls twice for each iteration of the for-loop, once for opening and once for closing the file descriptor associated with the file. This will become a bottle-neck on your performance for running large iterations. Recommended suggestion would be to move it outside the loop.

另一个重要的修复是将重定向移出 for 循环以减少大量文件 I/O。当在循环内使用时，shell 必须为 for 循环的每次迭代执行两次系统调用，一次用于打开，一次用于关闭与文件关联的文件描述符。这将成为运行大型迭代的性能瓶颈。推荐的建议是将它移到循环之外。

Extending the above code with this fixes, you could do

使用此修复程序扩展上述代码，您可以这样做

find . | xargs wc -l

which will basically put the contents of your command for each iteration of your file input to stdout and when the loop ends, open the target file once for writing the contents of the stdout and saving it. The equivalent findversion of the same would be

这基本上会将文件输入的每次迭代的命令内容放入标准输出，当循环结束时，打开目标文件一次以写入标准输出的内容并保存它。find相同的等效版本是

sh /dir/* > ./result.txt

回答by Eric Wooley

I needed to copy all .md files from one directory into another, so here is what I did.

我需要将所有 .md 文件从一个目录复制到另一个目录，所以这就是我所做的。

for i in **/*.md;do mkdir -p ../docs/"$i" && rm -r ../docs/"$i" && cp "$i" "../docs/$i" && echo "$i -> ../docs/$i"; done

for i in **/*.md;do mkdir -p ../docs/"$i" && rm -r ../docs/"$i" && cp "$i" "../docs/$i" && echo "$i -> ../docs/$i"; done

Which is pretty hard to read, so lets break it down.

这很难读，所以让我们分解一下。

first cd into the directory with your files,

首先 cd 进入包含您的文件的目录，

for i in **/*.md;for each file in your pattern

for i in **/*.md;对于模式中的每个文件

mkdir -p ../docs/"$i"make that directory in a docs folder outside of folder containing your files. Which creates an extra folder with the same name as that file.

mkdir -p ../docs/"$i"将该目录放在包含文件的文件夹之外的 docs 文件夹中。这会创建一个与该文件同名的额外文件夹。

rm -r ../docs/"$i"remove the extra folder that is created as a result of mkdir -p

rm -r ../docs/"$i"删除作为结果创建的额外文件夹 mkdir -p

cp "$i" "../docs/$i"Copy the actual file

cp "$i" "../docs/$i"复制实际文件

echo "$i -> ../docs/$i"Echo what you did

echo "$i -> ../docs/$i"回应你所做的

; doneLive happily ever after

; done从此幸福地生活

回答by Rahul

One quick and dirty way which gets the job done sometimes is:

有时完成工作的一种快速而肮脏的方法是：

$ export DIR=/path/dir && cd $DIR && chmod -R +x *
$ find . -maxdepth 1 -type f -name '*.sh' -exec {} \; > results.out

For example to find number of lines in all files in the current directory, you can do:

例如要查找当前目录中所有文件的行数，您可以执行以下操作：

$ export DIR=/path/dir && cd $DIR && chmod -R +x *
find . -maxdepth 2 -type f -name '*.sh' | sort | bash > results.out

回答by Al Mamun

You can use xarg

您可以使用 xarg

ls | xargs -L 1 -d '\n' your-desired-command

ls | xargs -L 1 -d '\n' your-desired-command

-L 1causes pass 1 item at a time

-L 1导致一次通过 1 个项目

-d '\n'make output of lsis split'ed based on new line.

-d '\n'ls根据新行拆分输出。

回答by yovie

i think the simple solution is:

我认为简单的解决方案是：

bash: 1: ./assets/main.sh
bash: 2: ./builder/clean.sh
bash: 3: ./builder/concept/compose.sh
bash: 4: ./builder/concept/market.sh
bash: 5: ./builder/concept/services.sh
bash: 6: ./builder/curl.sh
bash: 7: ./builder/identity.sh
bash: 8: ./concept/compose.sh
bash: 9: ./concept/market.sh
bash: 10: ./concept/services.sh
bash: 11: ./product/compose.sh
bash: 12: ./product/market.sh
bash: 13: ./product/services.sh
bash: 14: ./xferlog.sh

回答by Chetabahana

Maxdepth

最大深度

I found it works nicely with Jim Lewis's answerjust add a bit like this:

我发现它与吉姆刘易斯的答案很好地配合，只需添加一点如下：

export DIR=/path/dir && cd $DIR && chmod -R +x *
find . -type f -name '*.sh' | sort | bash > results.out

Sort Order

排序

If you want to execute in sort order, modify it like this:

如果要按排序顺序执行，可以这样修改：

#!/bin/bash
[[ "$(dirname `pwd`)" == $DIR ]] && echo "Executing `realpath 
if <a condition is matched>
then
    #execute child files
    export DIR=`pwd`
fi
`.." || return

Just for an example, this will execute with following order:

仅作为示例，这将按以下顺序执行：

$ find  directory/ -maxdepth 1 -type f -print0 | \
  xargs -r0 stat -c "%y %n" | \
  sort | cut -d' ' -f4- | \
  xargs -d "\n" -I{} cmd -op1 {} 

Unlimited Depth

无限深度

If you want to execute in unlimited depth by certain condition, you can use this:

如果你想在特定条件下无限深度地执行，你可以使用这个：

##代码##

then put on top of each files in the child directories like this:

然后像这样放在子目录中的每个文件的顶部：

##代码##

and somewhere in the body of parent file:

和父文件正文中的某处：

##代码##

回答by tuxdna

Based on @Jim Lewis's approach:

基于@Jim Lewis 的方法：

Here is a quick solution using findand also sorting files by their modification date:

这是使用find文件并按修改日期对文件进行排序的快速解决方案：

##代码##

For sorting see:

排序见：

http://www.commandlinefu.com/commands/view/5720/find-files-and-list-them-sorted-by-modification-time

http://www.commandlinefu.com/commands/view/5720/find-files-and-list-them-sorted-by-modification-time