Bash:使用 bash 脚本的 Head & Tail 行为
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/26461014/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Bash: Head & Tail behavior with bash script
提问by Mangat Rai Modi
Suppose I have following script:-
假设我有以下脚本:-
test.sh
测试文件
#!/bin/bash
command1 #prints 5 lines
command2 #prints 3 lines
I run the script with test.sh|head -n5
我运行脚本 test.sh|head -n5
What will happen in this case? Will it run both the commands? or will it stop after command1? What if I call it with -n1?
在这种情况下会发生什么?它会运行这两个命令吗?还是会在 command1 之后停止?如果我用-n1调用它会怎样?
Background:I might be asking a very basic question, but I actually noticed something interesting. My script(different one) was processing 7,000 files and each file produces 1 line of output. It takes 7 minutes to run the script completely but doing head -n1gave me prompt immediately like the script has terminated after processing first file only
背景:我可能会问一个非常基本的问题,但我实际上注意到了一些有趣的事情。我的脚本(不同的)正在处理 7,000 个文件,每个文件产生 1 行输出。完全运行脚本需要 7 分钟,但是执行head -n1立即给了我提示,就像脚本在仅处理第一个文件后终止一样
Edit:Following is my script
编辑:以下是我的脚本
for i in $(ls filepath);do
echo "$i" # issue here
python mySript "$i" > "/home/user/output/""$i"".out"
fi
done
Removing echoabove enables the script to run full 7 minute with head -n1, but with echo it just prints first line then exit.
删除上面的echo使脚本可以使用head -n1运行整整 7 分钟,但使用 echo 它只打印第一行然后退出。
采纳答案by TrueY
This is a fairly interesting issue! Thanks for posting it!
这是一个相当有趣的问题!感谢您发布它!
I assumed that this happens as head
exits after processing the first few lines, so SIGPIPE
signal is sent to the bashrunning the script when it tries to echo $x
next time. I used RedX's script to prove this theory:
我假设这是head
在处理前几行后退出时发生的,因此在下次尝试时将SIGPIPE
信号发送到运行脚本的bashecho $x
。我用 RedX 的脚本来证明这个理论:
#!/usr/bin/bash
rm x.log
for((x=0;x<5;++x)); do
echo $x
echo $x>>x.log
done
This works, as You described! Using t.sh|head -n 2
it writes only 2 lines to the screen and to x.log. But trapping SIGPIPE this behavior changes...
正如您所描述的,这有效!使用t.sh|head -n 2
它只会向屏幕和 x.log 写入 2 行。但是捕获 SIGPIPE 这种行为会改变......
#!/usr/bin/bash
trap "echo SIGPIPE>&2" PIPE
rm x.log
for((x=0;x<5;++x)); do
echo $x
echo $x>>x.log
done
Output:
输出:
$ ./t.sh |head -n 2
0
1
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
The write error occurs as stdout
is already closed as the other end of the pipe is closed. And any attempt to write to the closed pipe causes a SIGPIPE signal, which terminates the program by default (see man 7 signal
). The x.log now contains 5 lines.
当stdout
管道的另一端关闭时,写入错误已经关闭。任何写入关闭管道的尝试都会导致 SIGPIPE 信号,默认情况下会终止程序(请参阅 参考资料man 7 signal
)。x.log 现在包含 5 行。
This also explains why /bin/echo
solved the problem. See the following script:
这也解释了为什么/bin/echo
解决了这个问题。请参阅以下脚本:
rm x.log
for((x=0;x<5;++x)); do
/bin/echo $x
echo "Ret: $?">&2
echo $x>>x.log
done
Output:
输出:
$ ./t.sh |head -n 2
0
Ret: 0
1
Ret: 0
Ret: 141
Ret: 141
Ret: 141
Decimal 141 = hex 8D. Hex 80 means a signal was received, hex 0D is for SIGPIPE. So when /bin/echo
tried to write to stdout it got a SIGPIPE and it was terminated (as default behavior) instead of the bashrunning the script.
十进制 141 = 十六进制 8D。十六进制 80 表示接收到信号,十六进制 0D 表示 SIGPIPE。因此,当/bin/echo
尝试写入 stdout 时,它得到了一个 SIGPIPE 并且它被终止(作为默认行为)而不是运行脚本的bash。
回答by enrico.bacis
Nice finding. According to my tests it's exactly like you said. For example I have this script that just eats cpu, to let us spot it in top
:
不错的发现。根据我的测试,它和你说的完全一样。例如,我有一个只吃 CPU 的脚本,让我们发现它top
:
for i in `seq 10`
do echo $i
x=`seq 10000000`
done
Piping the script with head -n1
we see the command returning after the first line. This is the head
behavior: it completed its work, so it can stop and return the control to you.
管道脚本head -n1
我们看到命令在第一行之后返回。这是head
行为:它完成了它的工作,因此它可以停止并将控制权交还给您。
The input script should continue running but look what happens: when the head
returns, its piddoesn't exist anymore. So when linux tries to send the output of the script to the head process, it does not find the process, so the script crashes and stops.
输入脚本应该继续运行,但看看会发生什么:当head
返回时,它的pid不再存在。所以当linux尝试将脚本的输出发送到头进程时,它没有找到该进程,所以脚本崩溃并停止。
Let's try it with a python script:
让我们用一个 python 脚本来试试:
for i in xrange(10):
print i
range(10000000)
When running it and piping to head you have this:
当运行它并管道到头时,你有这个:
$ python -u test.py | head -n1
0
Traceback (most recent call last):
File "test.py", line 2, in <module>
print i
IOError: [Errno 32] Broken pipe
The -u
option tells python to automatically flush the stdin and stdout, as bash would do. So you see that the program actually stops with an error.
该-u
选项告诉 python 自动刷新标准输入和标准输出,就像 bash 所做的那样。所以你会看到程序实际上因错误而停止。
回答by RedX
This is more of a comment then an answer but it is too big for a comment.
这更像是评论而不是答案,但对于评论来说太大了。
I tried following script:
我尝试了以下脚本:
#!/usr/bin/env bash
rm -f "test_head.log"
echo "1 line"
echo "1 line" >> "test_head.log"
echo "2 line"
echo "2 line" >> "test_head.log"
echo "3 line"
echo "3 line" >> "test_head.log"
echo "4 line"
echo "4 line" >> "test_head.log"
echo "5 line"
echo "5 line" >> "test_head.log"
echo "6 line"
echo "6 line" >> "test_head.log"
echo "7 line"
echo "7 line" >> "test_head.log"
echo "8 line"
echo "8 line" >> "test_head.log"
Then i ran the script with:
然后我运行脚本:
./test_head.sh | head -n1
./test_head.sh | 头-n1
The cat output is (to my surprise):
cat 输出是(令我惊讶):
1 line
1 行
I have no idea what is going on.
我不知道发生了什么。
After reading @ymonad comment i tried it out and replace echo
with /bin/echo
and that solved the problem. I hope he can explain more about this behaviour.
阅读@ymonad 评论后,我试了一下并替换echo
为/bin/echo
,解决了问题。我希望他能更多地解释这种行为。