bash 你如何为文件的每一行运行一个命令？

Question

提问by hawk

For example, right now I'm using the following to change a couple of files whose Unix paths I wrote to a file:

例如，现在我正在使用以下内容来更改我将其 Unix 路径写入文件的几个文件：

cat file.txt | while read in; do chmod 755 "$in"; done

Is there a more elegant, safer way?

有没有更优雅、更安全的方法？

Answer 1

采纳答案by F. Hauri

Read a file line by line and execute commands: 4 answers

逐行读取文件并执行命令：4 个回答

This is because there is not only 1 answer...

这是因为不仅有 1 个答案...

shellcommand line expansion
xargsdedicated tool
while readwith some remarks
while read -uusing dedicated fd, for interactiveprocessing (sample)

shell命令行扩展
xargs专用工具
while read有一些评论
while read -u使用专用的fd，用于交互式处理（示例）

Regarding the OP request: running chmodon all targets listed in file, xargsis the indicated tool. But for some other applications, small amount of files, etc...

关于 OP 请求：在 file 中列出的所有目标上运行chmod，xargs是指示的工具。但是对于其他一些应用程序，少量文件等...

Read entire file as command line argument.
If your file is not too big and all files are well named(without spaces or other special chars like quotes), you could use shellcommand line expansion. Simply:
```
chmod 755 $(<file.txt)
```
For small amount of files (lines), this command is the lighter one.
xargsis the right tool
For bigger amount of files, or almost anynumber of lines in your input file...
For many binutilstools, like chown, chmod, rm, cp -t...
```
xargs chmod 755 <file.txt
```
If you have special chars and/or a lot of lines in file.txt.
```
xargs -0 chmod 755 < <(tr \n \0 <file.txt)
```
if your command need to be run exactly 1 time by entry:
```
xargs -0 -n 1 chmod 755 < <(tr \n \0 <file.txt)
```
This is not needed for this sample, as chmodaccept multiple files as argument, but this match the title of question.
For some special case, you could even define location of file argument in commands generateds by xargs:
```
xargs -0 -I '{}' -n 1 myWrapper -arg1 -file='{}' wrapCmd < <(tr \n \0 <file.txt)
```
Test with seq 1 5as input
Try this:
```
xargs -n 1 -I{} echo Blah {} blabla {}.. < <(seq 1 5)
Blah 1 blabla 1..
Blah 2 blabla 2..
Blah 3 blabla 3..
Blah 4 blabla 4..
Blah 5 blabla 5..
```
Where commande is done once per line.

`while read`and variants.

As OP suggest cat file.txt | while read in; do chmod 755 "$in"; donewill work, but there is 2 issues:

cat |is an useless fork, and
| while ... ;donewill become a subshellwhere environment will disapear after ;done.

So this could be better written:

while read in; do chmod 755 "$in"; done < file.txt

But,

You may be warned about $IFSand readflags:

help read

read: read [-r] ... [-d delim] ... [name ...]
    ...
    Reads a single line from the standard input... The line is split
    into fields as with word splitting, and the first word is assigned
    to the first NAME, the second word to the second NAME, and so on...
    Only the characters found in $IFS are recognized as word delimiters.
    ...
    Options:
      ...
      -d delim   continue until the first character of DELIM is read, 
                 rather than newline
      ...
      -r do not allow backslashes to escape any characters
    ...
    Exit Status:
    The return code is zero, unless end-of-file is encountered...

In some case, you may need to use

while IFS= read -r in;do chmod 755 "$in";done <file.txt

For avoiding problems with stranges filenames. And maybe if you encouter problems with UTF-8:

while LANG=C IFS= read -r in ; do chmod 755 "$in";done <file.txt

While you use STDINfor reading file.txt, your script could not be interactive(you cannot use STDINanymore).

while read -u, using dedicated fd.
Syntax: while read ...;done <file.txtwill redirect STDINto file.txt. That mean, you won't be able to deal with process, until they finish.
If you plan to create interactivetool, you have to avoid use of STDINand use some alternative file descriptor.
Constants file descriptorsare: 0for STDIN, 1for STDOUTand 2for STDERR. You could see them by:
```
ls -l /dev/fd/
```
or
```
ls -l /proc/self/fd/
```
From there, you have to choose unused number, between 0and 63(more, in fact, depending on sysctlsuperuser tool) as file descriptor:
For this demo, I will use fd7:
~~exec 7<file.txt # Without spaces between `7` and `<`! ls -l /dev/fd/~~

Then you could use read -u 7this way:
```
while read -u 7 filename;do
    ans=;while [ -z "$ans" ];do
        read -p "Process file '$filename' (y/n)? " -sn1 foo
        [ "$foo" ]&& [ -z "${foo/[yn]}" ]&& ans=$foo || echo '??'
    done
    if [ "$ans" = "y" ] ;then
        echo Yes
        echo "Processing '$filename'."
    else
        echo No
    fi
done 7<file.txt
```
done

To close fd/7:
exec 7<&- # This will close file descriptor 7. ls -l /dev/fd/

Nota: I let ~~striked~~version because this syntax could be usefull, when doing many I/O with parallels process:
```
mkfifo sshfifo
exec 7> >(ssh -t user@host sh >sshfifo)
exec 6<sshfifo
```

读取整个文件作为命令行参数。
如果您的文件不是太大，并且所有文件都很好命名（没有空格或其他特殊字符，如引号），您可以使用shell命令行扩展。简单地：
```
chmod 755 $(<file.txt)
```
对于少量文件（行），此命令是较轻的一个。
xargs是正确的工具
对于更大数量的文件，或输入文件中几乎任意数量的行...
对于许多binutils工具，例如chown, chmod, rm, cp -t...
```
xargs chmod 755 <file.txt
```
如果您有特殊字符和/或file.txt.
```
xargs -0 chmod 755 < <(tr \n \0 <file.txt)
```
如果您的命令需要按条目精确运行 1 次：
```
xargs -0 -n 1 chmod 755 < <(tr \n \0 <file.txt)
```
此示例不需要这，因为chmod接受多个文件作为参数，但这与问题的标题相匹配。
对于某些特殊情况，您甚至可以在通过xargs以下方式生成的命令中定义文件参数的位置：
```
xargs -0 -I '{}' -n 1 myWrapper -arg1 -file='{}' wrapCmd < <(tr \n \0 <file.txt)
```
使用seq 1 5作为输入进行测试
尝试这个：
```
xargs -n 1 -I{} echo Blah {} blabla {}.. < <(seq 1 5)
Blah 1 blabla 1..
Blah 2 blabla 2..
Blah 3 blabla 3..
Blah 4 blabla 4..
Blah 5 blabla 5..
```
每行执行一次命令。

`while read`和变种。

正如 OP 建议的那样cat file.txt | while read in; do chmod 755 "$in"; done，但有两个问题：

cat |是一个无用的叉子，并且
| while ... ;done将成为一个子外壳，其中环境将在;done.

所以这可以更好地写：

while read in; do chmod 755 "$in"; done < file.txt

但，

您可能会被警告$IFS和read标记：

help read

read: read [-r] ... [-d delim] ... [name ...]
    ...
    Reads a single line from the standard input... The line is split
    into fields as with word splitting, and the first word is assigned
    to the first NAME, the second word to the second NAME, and so on...
    Only the characters found in $IFS are recognized as word delimiters.
    ...
    Options:
      ...
      -d delim   continue until the first character of DELIM is read, 
                 rather than newline
      ...
      -r do not allow backslashes to escape any characters
    ...
    Exit Status:
    The return code is zero, unless end-of-file is encountered...

在某些情况下，您可能需要使用

while IFS= read -r in;do chmod 755 "$in";done <file.txt

用于避免奇怪文件名的问题。也许如果您遇到以下问题UTF-8：

while LANG=C IFS= read -r in ; do chmod 755 "$in";done <file.txt

当您STDIN用于阅读时file.txt，您的脚本无法进行交互（您无法再使用STDIN）。

while read -u，使用专用的fd。
语法：while read ...;done <file.txt将重定向STDIN到file.txt. 这意味着，在它们完成之前，您将无法处理流程。
如果您打算创建交互式工具，则必须避免使用STDIN和使用某些替代文件描述符。
常量文件描述符是：0用于STDIN，1用于STDOUT和2用于STDERR。您可以通过以下方式查看它们：
```
ls -l /dev/fd/
```
或者
```
ls -l /proc/self/fd/
```
从那里，您必须选择未使用的数字，介于0和之间63（更多，实际上取决于sysctl超级用户工具）作为文件描述符：
对于这个演示，我将使用fd7：
~~exec 7<file.txt # Without spaces between `7` and `<`! ls -l /dev/fd/~~

那么你可以这样使用read -u 7：
```
while read -u 7 filename;do
    ans=;while [ -z "$ans" ];do
        read -p "Process file '$filename' (y/n)? " -sn1 foo
        [ "$foo" ]&& [ -z "${foo/[yn]}" ]&& ans=$foo || echo '??'
    done
    if [ "$ans" = "y" ] ;then
        echo Yes
        echo "Processing '$filename'."
    else
        echo No
    fi
done 7<file.txt
```
done

关闭fd/7：
exec 7<&- # This will close file descriptor 7. ls -l /dev/fd/

注意：我使用了~~strike~~版本，因为在使用并行进程执行许多 I/O 时，此语法可能很有用：
```
mkfifo sshfifo
exec 7> >(ssh -t user@host sh >sshfifo)
exec 6<sshfifo
```

Answer 2

回答by P.P

Yes.

是的。

while read in; do chmod 755 "$in"; done < file.txt

This way you can avoid a catprocess.

这样你就可以避免一个cat过程。

catis almost always bad for a purpose such as this. You can read more about Useless Use of Cat.

cat对于这样的目的，几乎总是不好的。您可以阅读更多关于无用使用 Cat 的信息。

Answer 3

回答by d.raev

if you have a nice selector (for example all .txt files in a dir) you could do:

如果您有一个不错的选择器（例如目录中的所有 .txt 文件），您可以执行以下操作：

for i in *.txt; do chmod 755 "$i"; done

bash for loop

bash for 循环

or a variant of yours:

或您的变体：

while read line; do chmod 755 "$line"; done <file.txt

Answer 4

回答by glenn Hymanman

If you know you don't have any whitespace in the input:

如果您知道输入中没有任何空格：

xargs chmod 755 < file.txt

If there might be whitespace in the paths, and if you have GNU xargs:

如果路径中可能有空格，并且您有 GNU xargs：

tr '\n' 'parallel -a <your file> <program>
' < file.txt | xargs -0 chmod 755

Answer 5

回答by janisz

If you want to run your command in parallel for each line you can use GNU Parallel

如果你想为每一行并行运行你的命令，你可以使用GNU Parallel

perl -p -e '`chmod 755 $_`' file.txt

Each line of your file will be passed to program as an argument. By default parallelruns as many threads as your CPUs count. But you can specify it with -j

文件的每一行都将作为参数传递给程序。默认情况下，parallel运行与 CPU 数量一样多的线程。但是你可以指定它-j

Answer 6

回答by 1.618

I see that you tagged bash, but Perl would also be a good way to do this:

我看到你标记了 bash，但 Perl 也是一个很好的方法：

perl -p -e 'if(/\.txt$/) `chmod 755 $_`' file.txt

You could also apply a regex to make sure you're getting the right files, e.g. to only process .txt files:

您还可以应用正则表达式来确保您获得正确的文件，例如只处理 .txt 文件：

perl -p -e 'if(/\.txt$/) print "chmod 755 $_"' file.txt

To "preview" what's happening, just replace the backticks with double quotes and prepend print:

要“预览”正在发生的事情，只需用双引号替换反引号并添加print：

awk '{ print "chmod 755 "awk -F, '{ print "chmod 755 """ | "/bin/sh"}' file.txt
"" | "/bin/sh"}' file.txt

Answer 7

回答by brunocrt

You can also use AWK which can give you more flexibility to handle the file

您还可以使用 AWK，它可以让您更灵活地处理文件

#!/bin/ksh
last |head -10|awk '{print }'|
 while IFS= read -r line
 do
su - "$line" -c 'tail .sh_history'
 done

if your file has a field separator like:

如果您的文件有一个字段分隔符，如：

field1,field2,field3

字段 1、字段 2、字段 3

To get only the first field you do

只获得你做的第一个字段

cat file | tr -d '\r' | xargs -L 1 -i echo do_sth_with_{}_as_line

You can check more details on GNU Documentation https://www.gnu.org/software/gawk/manual/html_node/Very-Simple.html#Very-Simple

您可以查看有关 GNU 文档的更多详细信息 https://www.gnu.org/software/gawk/manual/html_node/Very-Simple.html#Very-Simple

Answer 8

回答by brunocrt

The logic applies to many other objectives. And how to read .sh_history of each user from /home/ filesystem? What if there are thousand of them?

该逻辑适用于许多其他目标。以及如何从 /home/ 文件系统读取每个用户的 .sh_history ？如果有几千个呢？

##代码##

Here is the script https://github.com/imvieira/SysAdmin_DevOps_Scripts/blob/master/get_and_run.sh

这是脚本https://github.com/imvieira/SysAdmin_DevOps_Scripts/blob/master/get_and_run.sh

Answer 9

回答by E P

I know it's late but still

我知道已经晚了但仍然

If by any chance you run into windows saved text file with \r\ninstead of \n, you might get confused by the output if your command has sth after read line as argument. So do remove \r, for example:

如果您遇到 Windows 保存的文本文件\r\n而不是\n，如果您的命令在 read 行之后作为参数，您可能会对输出感到困惑。所以做 remove \r，例如：

##代码##

bash 你如何为文件的每一行运行一个命令？

提问by hawk

采纳答案by F. Hauri

Read a file line by line and execute commands: 4 answers

逐行读取文件并执行命令：4 个回答

Read entire file as command line argument.

`xargs`is the right tool

Test with `seq 1 5`as input

`while read`and variants.

`while read -u`, using dedicated `fd`.

读取整个文件作为命令行参数。

`xargs`是正确的工具

使用`seq 1 5`作为输入进行测试

`while read`和变种。

`while read -u`，使用专用的`fd`。

回答by P.P

回答by d.raev

回答by glenn Hymanman

回答by janisz

回答by 1.618

回答by brunocrt

回答by brunocrt

回答by E P

相关推荐

最近更新

标签

bash 你如何为文件的每一行运行一个命令？

提问by hawk

采纳答案by F. Hauri

Read a file line by line and execute commands: 4 answers

逐行读取文件并执行命令：4 个回答

Read entire file as command line argument.

xargsis the right tool

Test with seq 1 5as input

while readand variants.

while read -u, using dedicated fd.

读取整个文件作为命令行参数。

xargs是正确的工具

使用seq 1 5作为输入进行测试

while read和变种。

while read -u，使用专用的fd。

回答by P.P

回答by d.raev

回答by glenn Hymanman

回答by janisz

回答by 1.618

回答by brunocrt

回答by brunocrt

回答by E P

相关推荐

bash 添加到 .bashrc 文件的路径？

bash 使用 shell 脚本中的内容创建文件

bash 在 Linux shell 脚本中，如何打印数组的最大值和最小值？

如何在 Bash 中找到数组元素的总和？

相关推荐

最近更新

标签

`xargs`is the right tool

Test with `seq 1 5`as input

`while read`and variants.

`while read -u`, using dedicated `fd`.

`xargs`是正确的工具

使用`seq 1 5`作为输入进行测试

`while read`和变种。

`while read -u`，使用专用的`fd`。