bash 输出到文件时的 GNU 并行输出进度

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/18837854/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 06:35:23  来源:igfitidea点击:

GNU parallel output progress while output to file

bashparallel-processinggnu-parallel

提问by Alvin Das

I have a simple bash script to run:

我有一个简单的 bash 脚本可以运行:

cat full_path.csv | parallel --progress -j +0 'echo -n {},; pdfgrep -c [^_] {};' > path_count.csv

Parallel's progress indicator "--progress", writes into the file path_count.csv. I only want echo {}and pdfgrep {}to write to the file, while showing --progressoutput to screen.

Parallel 的进度指示器"--progress",写入文件path_count.csv。我只想echo {}pdfgrep {}写入文件,同时显示--progress到屏幕输出。

If I do :

如果我做 :

cat full_path.csv | parallel --progress -j +0  'echo -n {},>>path_count.csv; pdfgrep -c [^_] {}>>path_count.csv;'

the file path_count is still garbled with progress.

文件 path_count 仍然是乱码。

Any help is appreciated. Thanks Alvin

任何帮助表示赞赏。谢谢阿尔文

采纳答案by Ole Tange

The behaviour you see is not what GNU Parallel is designed to do: --progress is normally sent to STDERR and not to STDOUT for exactly that reason:

您看到的行为不是 GNU Parallel 的设计目的:--progress 通常被发送到 STDERR 而不是 STDOUT 正是出于这个原因:

$ seq 3 | bin/parallel --progress echo {} >/tmp/out

Computers / CPU cores / Max jobs to run
1:local / 8 / 3

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
local:0/3/100%/0.0s
$ cat /tmp/out
1
2
3

Has there been local modifications of GNU Parallel? Can you reproduce the issue on other systems?

是否对 GNU Parallel 进行了本地修改?你能在其他系统上重现这个问题吗?

PS: instead of 'echo -n' why to try: --tag

PS:而不是 'echo -n' 为什么要尝试:--tag

回答by konsolebox

Try to redirect it completely within the subshell using exec:

尝试使用 exec 在子shell中完全重定向它:

: > path_count.csv  ## truncate file
cat full_path.csv | parallel --progress -j +0 'exec >>path_count.csv; echo -n {},; pdfgrep -c [^_] {};'