如何在 Bash 中区分两个管道？

How can you diff two pipelines in Bash?

提问by Adam Rosenfield

How can you difftwo pipelines without using temporary files in Bash? Say you have two command pipelines:

如何在不使用 Bash 中的临时文件的情况下区分两个管道？假设您有两个命令管道：

foo | bar
baz | quux

And you want to find the diffin their outputs. One solution would obviously be to:

你想diff在他们的输出中找到。一种解决方案显然是：

foo | bar > /tmp/a
baz | quux > /tmp/b
diff /tmp/a /tmp/b

Is it possible to do so without the use of temporary files in Bash? You can get rid of one temporary file by piping in one of the pipelines to diff:

在 Bash 中不使用临时文件是否可以这样做？您可以通过管道之一来删除一个临时文件来区分：

foo | bar > /tmp/a
baz | quux | diff /tmp/a -

But you can't pipe both pipelines into diff simultaneously (not in any obvious manner, at least). Is there some clever trick involving /dev/fdto do this without using temporary files?

但是您不能同时将两个管道连接到 diff 中（至少不是以任何明显的方式）。是否有一些巧妙的技巧/dev/fd可以在不使用临时文件的情况下做到这一点？

回答by VonC

A one-line with 2 tmp files (not what you want) would be:

一行包含 2 个 tmp 文件（不是您想要的）将是：

 foo | bar > file1.txt && baz | quux > file2.txt && diff file1.txt file2.txt

With bash, you might try though:

使用bash，您可以尝试：

 diff <(foo | bar) <(baz | quux)

 foo | bar | diff - <(baz | quux)  # or only use process substitution once

The 2nd version will more clearly remind you which input was which, by showing
-- /dev/stdinvs. ++ /dev/fd/63or something, instead of two numbered fds.

第二个版本会更清楚地提醒你哪个输入是哪个，通过显示
-- /dev/stdinvs.++ /dev/fd/63或其他东西，而不是两个编号的 fds。

---

---

Not even a named pipe will appear in the filesystem, at least on OSes where bash can implement process substitution by using filenames like /dev/fd/63to get a filename that the command can open and read from to actually read from an already-open file descriptor that bash set up before exec'ing the command. (i.e. bash uses pipe(2)before fork, and then dup2to redirect from the output of quuxto an input file descriptor for diff, on fd 63.)

甚至命名管道都不会出现在文件系统中，至少在 bash 可以通过使用文件名来实现进程替换的操作系统上，例如/dev/fd/63获取命令可以打开和读取的文件名，以实际从 bash 设置的已经打开的文件描述符中读取在执行命令之前启动。（即 bashpipe(2)在 fork 之前使用，然后在 fd 63 上dup2从 的输出重定向quux到输入文件描述符diff。）

On a system with no "magical" /dev/fdor /proc/self/fd, bash might use named pipes to implement process substitution, but it would at least manage them itself, unlike temporary files, and your data wouldn't be written to the filesystem.

在没有“神奇”/dev/fd或/proc/self/fd.

You can check how bash implements process substitution with echo <(true)to print the filename instead of reading from it. It prints /dev/fd/63on a typical Linux system. Or for more details on exactly what system calls bash uses, this command on a Linux system will trace file and file-descriptor system calls

您可以检查 bash 如何实现进程替换echo <(true)以打印文件名而不是从中读取文件名。它/dev/fd/63在典型的 Linux 系统上打印。或者有关 bash 使用的系统调用的更多详细信息，Linux 系统上的此命令将跟踪文件和文件描述符系统调用

strace -f -efile,desc,clone,execve bash -c '/bin/true | diff -u - <(/bin/true)'

---

---

Without bash, you could make a named pipe. Use -to tell diffto read one input from STDIN, and use the named pipe as the other:

没有 bash，你可以创建一个命名管道。使用-告诉diff从标准输入读一个输入，并使用命名管道作为其他：

mkfifo file1_pipe.txt
foo|bar > file1_pipe.txt && baz | quux | diff file1_pipe.txt - && rm file1_pipe.txt

---

---

Note that you can only pipe one outputto multiple inputswith the tee command:

请注意，您只能使用 tee 命令将一个输出通过管道传输到多个输入：

ls *.txt | tee /dev/tty txtlist.txt 

The above command displays the output of ls *.txt to the terminal and outputs it to the text file txtlist.txt.

上述命令将ls *.txt的输出显示到终端，并输出到文本文件txtlist.txt。

But with process substitution, you can use teeto feed the same data into multiple pipelines:

但是通过流程替换，您可以使用tee将相同的数据提供给多个管道：

cat *.txt | tee >(foo | bar > result1.txt)  >(baz | quux > result2.txt) | foobar

回答by BenM

In bash you can use subshells, to execute the command pipelines individually, by enclosing the pipeline within parenthesis. You can then prefix these with < to create anonymous named pipes which you can then pass to diff.

在 bash 中，您可以使用子外壳程序，通过将管道括在括号内来单独执行命令管道。然后，您可以使用 < 作为前缀来创建匿名命名管道，然后您可以将其传递给 diff。

For example:

例如：

diff <(foo | bar) <(baz | quux)

The anonymous named pipes are managed by bash so they are created and destroyed automatically (unlike temporary files).

匿名命名管道由 bash 管理，因此它们会自动创建和销毁（与临时文件不同）。

回答by mlg

Some people arriving at this page might be looking for a line-by-line diff, for which commor grep -fshould be used instead.

到达此页面的某些人可能正在寻找逐行差异，为此应使用comm或grep -f代替。

One thing to point out is that, in all of the answer's examples, the diffs won't actually start until both streams have finished. Test this with e.g.:

需要指出的一件事是，在所有答案的示例中，直到两个流都完成后，差异才会真正开始。用例如测试这个：

comm -23 <(seq 100 | sort) <(seq 10 20 && sleep 5 && seq 20 30 | sort)

If this is an issue, you could try sd(stream diff), which doesn't require sorting (like commdoes) nor process substitution like the above examples, is orders or magnitude faster than grep -fand supports infinite streams.

如果这是一个问题，您可以尝试sd(stream diff)，它不需要排序（就像comm上面的例子那样）或处理替换，比grep -f并支持无限流快的数量级或数量级。

The test example I propose would be written like this in sd:

我建议的测试示例将这样编写sd：

seq 100 | sd 'seq 10 20 && sleep 5 && seq 20 30'

But the difference is that seq 100would be diffed with seq 10right away. Note that, if one of the streams is a tail -f, the diff cannot be done with process substitution.

但不同的是，马上seq 100就会有所不同seq 10。请注意，如果其中一个流是 a tail -f，则无法通过进程替换来完成差异。

Here's a blogpostI wrote about diffing streams on the terminal, which introduces sd.

这是我写的一篇关于在终端上区分流的博客文章，它介绍了sd.