PipedInputStream - 如何避免“java.io.IOException:管道损坏”

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/1866255/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-12 23:23:39  来源:igfitidea点击:

PipedInputStream - How to avoid "java.io.IOException: Pipe broken"

javamultithreadingio

提问by levanovd

I have two threads. One of them writes to PipedOutputStream, another one reads from corresponding PipedInputStream. Background is that one thread is downloading some data from remote server and multiplexes it to several other threads through piped streams.

我有两个线程。其中一个写入 PipedOutputStream,另一个从相应的 PipedInputStream 读取。背景是一个线程正在从远程服务器下载一些数据,并通过管道流将其多路复用到其他几个线程。

The problem is that sometimes (especially when downloading large (>50Mb)files) I get java.io.IOException: Pipe brokenwhen trying to read from PipedInputStream.
Javadoc says that A pipe is said to be broken if a thread that was providing data bytes to the connected piped output stream is no longer alive.
It is true, my writing thread really dies after writing all his data to PipedOutputStream.

问题是有时(尤其是在下载大型(> 50Mb)文件时)我会在尝试从 PipedInputStream 读取时得到java.io.IOException: Pipebreak 。
Javadoc 说A pipe is said to be broken if a thread that was providing data bytes to the connected piped output stream is no longer alive.
这是真的,我的写作线程在将他的所有数据写入 PipedOutputStream 后真的死了。

Any solutions? How can I prevent PipedInputStream from throwing this exception? I want to be able to read all data that was written to PipedOutputStream even if writing thread finished his work. (If anybody knows how to keep writing thread alive until all data will be read, this solution is also acceptable).

任何解决方案?如何防止 PipedInputStream 抛出此异常?即使写线程完成了他的工作,我也希望能够读取所有写入 PipedOutputStream 的数据。(如果有人知道如何在读取所有数据之前保持写入线程处于活动状态,则此解决方案也是可以接受的)。

采纳答案by Jerome

Use a java.util.concurrent.CountDownLatch, and do not end the first thread before the second one has signaled that is has finished reading from the pipe.

使用 java.util.concurrent.CountDownLatch,并且在第二个线程表示已完成从管道读取之前不要结束第一个线程。

Update: quick and dirty code to illustrate my comment below

更新:快速而肮脏的代码来说明我在下面的评论

    final PipedInputStream pin = getInputStream();
    final PipedOutputStream pout = getOutputStream();

    final CountDownLatch latch = new CountDownLatch(1);

    InputStream in = new InputStream() {

        @Override
        public int read() throws IOException {
            return pin.read();
        }

        @Override
        public void close() throws IOException {
            super.close();
            latch.countDown();
        }
    };


    OutputStream out = new OutputStream(){

        @Override
        public void write(int b) throws IOException {
            pout.write(b);
        }

        @Override
        public void close() throws IOException {
            while(latch.getCount()!=0) {
                try {
                    latch.await();
                } catch (InterruptedException e) {
                    //too bad
                }
            }
            super.close();
        }
    };

    //give the streams to your threads, they don't know a latch ever existed
    threadOne.feed(in);
    threadTwo.feed(out);

回答by Tom Hawtin - tackline

PipedInputStreamand PipedOutputStreamare broken (with regards to threading). They assume each instance is bound to a particular thread. This is bizarre. I suggest using your own (or at least a different) implementation.

PipedInputStream并且PipedOutputStream坏了(关于穿线)。他们假设每个实例都绑定到一个特定的线程。这很奇怪。我建议使用您自己的(或至少是不同的)实现。

回答by wds

Are you closing your PipedOutputStreamwhen the thread that's using it ends? You need to do this so the bytes in it will get flushed to the corresponding PipedInputStream.

PipedOutputStream当使用它的线程结束时,您是否关闭了?您需要这样做,以便将其中的字节刷新到相应的PipedInputStream.

回答by Marcono1234

You might encounter problems with these classes when you use more than one reader or writer thread (see JDK-4028322).

当您使用多个读取器或写入器线程时,您可能会遇到这些类的问题(请参阅JDK-4028322)。

However most users likely only use one reader and one writer thread. Since this is the case for you as well, the reason why you are encountering a broken pipe is most likely that you did not closethe PipedOutputStreamonce you are done writing.

然而,大多数用户可能只使用一个读取器和一个写入器线程。由于这是你的情况下,为什么你遇到一个破裂的管道的原因最有可能是你没有closePipedOutputStream,一旦你写完。

PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream(out);

// You can of course also use an Executor or similar for this
new Thread(() -> {
    // Your method writing the data
    writeDataTo(out);
    // Close PipedOutputStream so that reader knows that the end of the data 
    // has been reached
    try {
        out.close();
    }
    catch (IOException e) {
        // Handle exception or simply ignore it; current implementation will NEVER 
        // throw an IOException: https://github.com/openjdk/jdk/blob/0e14d5a98453407488057e6714f90f04d7dfa383/src/java.base/share/classes/java/io/PipedOutputStream.java#L174
    }
}).start();

// Your method reading the data
readDataFrom(in);
// Close PipedInputStream so that writer fails instead of blocking infinitely in case 
// it tries to write again (for whatever reason)
in.close();

There is also no need to manually call PipedOutputStream.flush(), it only notifies waiting readers, but no data is lost if you directly call close().

也不需要手动调用PipedOutputStream.flush(),只通知等待的读者,直接调用不会丢失数据close()

Sadly the documentation is at the moment not very clear about all of this. In general it is not a good idea to rely on the implementation, but in this case it might be the only sensible solution:

遗憾的是,文档目前对所有这些都不是很清楚。一般来说,依赖实现不是一个好主意,但在这种情况下,它可能是唯一明智的解决方案: