Bash 脚本不会继续读取文件的下一行

Bash script does not continue to read the next line of file

提问by Abs

I have a shell script that saves the output of a command that is executed to a CSV file. It reads the command it has to execute from a shell script which is in this format:

我有一个 shell 脚本，可以将执行的命令的输出保存到 CSV 文件中。它从以下格式的 shell 脚本中读取它必须执行的命令：

ffmpeg -i /home/test/videos/avi/418kb.avi /home/test/videos/done/418kb.flv
ffmpeg -i /home/test/videos/avi/1253kb.avi /home/test/videos/done/1253kb.flv
ffmpeg -i /home/test/videos/avi/2093kb.avi /home/test/videos/done/2093kb.flv

You can see each line is an ffmpeg command. However, the script just executes the first line. Just a minute ago it was doing nearly all of the commands. It was missing half for some reason. I edited the text file that contained the commands and now it will only do the first line. Here is my bash script:

你可以看到每一行都是一个 ffmpeg 命令。但是，脚本只执行第一行。就在一分钟前，它正在执行几乎所有的命令。由于某种原因，它丢失了一半。我编辑了包含命令的文本文件，现在它只会执行第一行。这是我的 bash 脚本：

#!/bin/bash
# Shell script utility to read a file line line.
# Once line is read it will run processLine() function

#Function processLine
processLine(){
line="$@"

START=$(date +%s.%N)

eval $line > /dev/null 2>&1 

END=$(date +%s.%N)
DIFF=$(echo "$END - $START" | bc)

echo "$line, $START, $END, $DIFF" >> file.csv 2>&1
echo "It took $DIFF seconds"
echo $line
}

# Store file name
FILE=""

# get file name as command line argument
# Else read it from standard input device
if [ "" == "" ]; then
   FILE="/dev/stdin"
else
   FILE=""
   # make sure file exist and readable
   if [ ! -f $FILE ]; then
    echo "$FILE : does not exists"
    exit 1
   elif [ ! -r $FILE ]; then
    echo "$FILE: can not read"
    exit 2
   fi
fi
# read $FILE using the file descriptors

# Set loop separator to end of line
BAKIFS=$IFS
IFS=$(echo -en "\n\b")
exec 3<&0
exec 0<$FILE
while read line
do
    # use $line variable to process line in processLine() function
    processLine $line
done
exec 0<&3

# restore $IFS which was used to determine what the field separators are
BAKIFS=$ORIGIFS
exit 0

Thank you for any help.

感谢您的任何帮助。

UPDATE 2

更新 2

Its the ffmpeg commands rather than the shell script that isn't working. But I should of been using just "\b" as Paul pointed out. I am also making use of Johannes's shorter script.

它是 ffmpeg 命令而不是不起作用的 shell 脚本。但正如保罗指出的那样，我应该只使用“\b”。我也在使用 Johannes 较短的脚本。

回答by Johannes Weiss

I think that should do the same and seems to be correct:

我认为应该这样做并且似乎是正确的：

#!/bin/bash

CSVFILE=/tmp/file.csv

cat "$@" | while read line; do
    echo "Executing '$line'"
    START=$(date +%s)
    eval $line &> /dev/null
    END=$(date +%s)
    let DIFF=$END-$START

    echo "$line, $START, $END, $DIFF" >> "$CSVFILE"
    echo "It took ${DIFF}s"
done

no?

不？

回答by mivk

ffmpeg reads STDIN and exhausts it. The solution is to call ffmpeg with:

ffmpeg 读取 STDIN 并将其耗尽。解决方案是调用 ffmpeg ：

 ffmpeg </dev/null ...

See the detailed explanation here: http://mywiki.wooledge.org/BashFAQ/089

请参阅此处的详细说明：http: //mywiki.wooledge.org/BashFAQ/089

回答by KPMueller

I just had the same problem.

我只是遇到了同样的问题。

I believe ffmpeg is responsible for this behaviour.

我相信 ffmpeg 应对这种行为负责。

My solution for this problem:

我对这个问题的解决方案：

1) Call ffmpeg with an "&" at the end of your ffmpeg command line

1) 在 ffmpeg 命令行的末尾使用“&”调用 ffmpeg

2) Since now the skript will not wait till completion of the ffmpeg process, we have to prevent our script from starting several ffmpeg processes. We achieve this goal by delaying the loop pass while there is at least one running ffmpeg process.

2）由于现在脚本不会等到ffmpeg进程完成，我们必须防止我们的脚本启动几个ffmpeg进程。我们通过在至少有一个正在运行的 ffmpeg 进程时延迟循环来实现这一目标。

#!/bin/bash

cat FileList.txt |
while read VideoFile; do
    <place your ffmpeg command line here> &
    FFMPEGStillRunning="true"
    while [ "$FFMPEGStillRunning" = "true" ]; do
        Process=$(ps -C ffmpeg | grep -o -e "ffmpeg" )
        if [ -n "$Process" ]; then
            FFMPEGStillRunning="true"
        else
            FFMPEGStillRunning="false"
        fi 
        sleep 2s
    done
done

回答by Paul Tomblin

I would add echos before and after the eval to see what it's about to eval (in case it's treating the whole file as one big long line) and after (in case one of the ffmpeg commands is taking forever).

我会在 eval 之前和之后添加 echo 以查看它将要评估的内容（以防它将整个文件视为一个大长行）和之后（以防 ffmpeg 命令之一永远执行）。

回答by Jonathan Leffler

Unless you are planning to read something from standard input after the loop, you don't need to preserve and restore the original standard input (though it is good to see you know how).

除非您计划在循环后从标准输入读取某些内容，否则您不需要保留和恢复原始标准输入（尽管很高兴看到您知道如何操作）。

Similarly, I don't see a reason for dinking with IFS at all. There is certainly no need to restore the value of IFS before exit - this is a real shell you are using, not a DOS BAT file.

同样，我根本看不到与 IFS 打交道的理由。退出前肯定不需要恢复IFS的值——这是你使用的真实shell，不是DOS BAT文件。

When you do:

当你这样做时：

read var1 var2 var3

the shell assigns the first field to $var1, the second to $var2, and the rest of the line to $var3. In the case where there's just one variable - your script, for example - the whole line goes into the variable, just as you want it to.

shell 将第一个字段分配给$var1，将第二个字段分配给$var2，并将行的其余部分分配给$var3。在只有一个变量的情况下——例如你的脚本——整行都进入变量，就像你想要的那样。

Inside the process line function, you probably don't want to throw away error output from the executed command. You probably do want to think about checking the exit status of the command. The echowith error redirection is ... unusual, and overkill. If you're sufficiently sure that the commands can't fail, then go ahead with ignoring the error. Is the command 'chatty'; if so, throw away the chat by all means. If not, maybe you don't need to throw away standard output, either.

在流程行函数中，您可能不想丢弃已执行命令的错误输出。您可能确实想考虑检查命令的退出状态。该echo错误重定向...不寻常的，并矫枉过正。如果您足够确定命令不会失败，则继续忽略错误。命令是“健谈”吗？如果是这样，一定要扔掉聊天。如果没有，也许您也不需要丢弃标准输出。

The script as a whole should probably diagnose when it is given multiple files to process since it ignores the extraneous ones.

脚本作为一个整体可能应该在给它处理多个文件时进行诊断，因为它会忽略无关的文件。

You could simplify your file handling by using just:

您可以仅使用以下内容来简化文件处理：

cat "$@" |
while read line
do
    processline "$line"
done

The catcommand automatically reports errors (and continues after them) and processes all the input files, or reads standard input if there are no arguments left. The use of double quotes around the variable means that it is passed as a single unit (and therefore unparsed into separate words).

该cat命令会自动报告错误（并在它们之后继续）并处理所有输入文件，或者如果没有剩余参数则读取标准输入。在变量周围使用双引号意味着它作为单个单元传递（因此未解析为单独的单词）。

The use of dateand bcis interesting - I'd not seen that before.

使用的date和bc有趣的是-我没有看到过。

All in all, I'd be looking at something like:

总而言之，我会看类似的东西：

#!/bin/bash
# Time execution of commands read from a file, line by line.
# Log commands and times to CSV logfile "file.csv"

processLine(){
    START=$(date +%s.%N)
    eval "$@" > /dev/null
    STATUS=$?
    END=$(date +%s.%N)
    DIFF=$(echo "$END - $START" | bc)
    echo "$line, $START, $END, $DIFF, $STATUS" >> file.csv
    echo "${DIFF}s: $STATUS: $line"
}

cat "$@" |
while read line
do
    processLine "$line"
done