如何在 C++ 中编写自定义输入流

How to write custom input stream in C++

提问by kayahr

I'm currently learning C++ (Coming from Java) and I'm trying to understand how to use IO streams properly in C++.

我目前正在学习 C++（来自 Java）并且我正在尝试了解如何在 C++ 中正确使用 IO 流。

Let's say I have an Imageclass which contains the pixels of an image and I overloaded the extraction operator to read the image from a stream:

假设我有一个Image包含图像像素的类，并且我重载了提取运算符以从流中读取图像：

istream& operator>>(istream& stream, Image& image)
{
    // Read the image data from the stream into the image
    return stream;
}

So now I'm able to read an image like this:

所以现在我可以读取这样的图像：

Image image;
ifstream file("somepic.img");
file >> image;

But now I want to use the same extraction operator to read the image data from a custom stream. Let's say I have a file which contains the image in compressed form. So instead of using ifstream I might want to implement my own input stream. At least that's how I would do it in Java. In Java I would write a custom class extending the InputStreamclass and implementing the int read()method. So that's pretty easy. And usage would look like this:

但现在我想使用相同的提取运算符从自定义流中读取图像数据。假设我有一个包含压缩形式图像的文件。因此，我可能想实现自己的输入流，而不是使用 ifstream。至少这就是我在 Java 中的做法。在 Java 中，我将编写一个扩展InputStream类并实现该int read()方法的自定义类。所以这很容易。用法如下所示：

InputStream stream = new CompressedInputStream(new FileInputStream("somepic.imgz"));
image.read(stream);

So using the same pattern maybe I want to do this in C++:

所以使用相同的模式也许我想在 C++ 中做到这一点：

Image image;
ifstream file("somepic.imgz");
compressed_stream stream(file);
stream >> image;

But maybe that's the wrong way, don't know. Extending the istreamclass looks pretty complicated and after some searching I found some hints about extending streambufinstead. But this examplelooks terribly complicated for such a simple task.

但也许那是错误的方式，不知道。扩展istream类看起来很复杂，经过一番搜索，我发现了一些关于扩展的提示streambuf。但是对于这样一个简单的任务，这个例子看起来非常复杂。

So what's the best way to implement custom input/output streams (or streambufs?) in C++?

那么在 C++ 中实现自定义输入/输出流（或流缓冲？）的最佳方法是什么？

Solution

解决方案

Some people suggested not using iostreams at all and to use iterators, boost or a custom IO interface instead. These may be valid alternatives but my question was about iostreams. The accepted answer resulted in the example code below. For easier reading there is no header/code separation and the whole std namespace is imported (I know that this is a bad thing in real code).

有些人建议根本不要使用 iostreams，而是使用迭代器、boost 或自定义 IO 接口。这些可能是有效的替代方案，但我的问题是关于 iostreams。接受的答案产生了下面的示例代码。为了更容易阅读，没有标题/代码分离，并且导入了整个 std 命名空间（我知道这在实际代码中是一件坏事）。

This example is about reading and writing vertical-xor-encoded images. The format is pretty easy. Each byte represents two pixels (4 bits per pixel). Each line is xor'd with the previous line. This kind of encoding prepares the image for compression (usually results in lot of 0-bytes which are easier to compress).

这个例子是关于读取和写入垂直异或编码的图像。格式很简单。每个字节代表两个像素（每像素 4 位）。每行都与前一行异或。这种编码为压缩图像做准备（通常会产生很多更容易压缩的 0 字节）。

#include <cstring>
#include <fstream>

using namespace std;

/*** vxor_streambuf class ******************************************/

class vxor_streambuf: public streambuf
{
public:
    vxor_streambuf(streambuf *buffer, const int width) :
        buffer(buffer),
        size(width / 2)
    {
        previous_line = new char[size];
        memset(previous_line, 0, size);
        current_line = new char[size];
        setg(0, 0, 0);
        setp(current_line, current_line + size);
    }

    virtual ~vxor_streambuf()
    {
        sync();
        delete[] previous_line;
        delete[] current_line;
    }

    virtual streambuf::int_type underflow()
    {
        // Read line from original buffer
        streamsize read = buffer->sgetn(current_line, size);
        if (!read) return traits_type::eof();

        // Do vertical XOR decoding
        for (int i = 0; i < size; i += 1)
        {
            current_line[i] ^= previous_line[i];
            previous_line[i] = current_line[i];
        }

        setg(current_line, current_line, current_line + read);
        return traits_type::to_int_type(*gptr());
    }

    virtual streambuf::int_type overflow(streambuf::int_type value)
    {
        int write = pptr() - pbase();
        if (write)
        {
            // Do vertical XOR encoding
            for (int i = 0; i < size; i += 1)
            {
                char tmp = current_line[i];
                current_line[i] ^= previous_line[i];
                previous_line[i] = tmp;
            }

            // Write line to original buffer
            streamsize written = buffer->sputn(current_line, write);
            if (written != write) return traits_type::eof();
        }

        setp(current_line, current_line + size);
        if (!traits_type::eq_int_type(value, traits_type::eof())) sputc(value);
        return traits_type::not_eof(value);
    };

    virtual int sync()
    {
        streambuf::int_type result = this->overflow(traits_type::eof());
        buffer->pubsync();
        return traits_type::eq_int_type(result, traits_type::eof()) ? -1 : 0;
    }

private:
    streambuf *buffer;
    int size;
    char *previous_line;
    char *current_line;
};


/*** vxor_istream class ********************************************/

class vxor_istream: public istream
{
public:
    vxor_istream(istream &stream, const int width) :
        istream(new vxor_streambuf(stream.rdbuf(), width)) {}

    virtual ~vxor_istream()
    {
        delete rdbuf();
    }
};


/*** vxor_ostream class ********************************************/

class vxor_ostream: public ostream
{
public:
    vxor_ostream(ostream &stream, const int width) :
        ostream(new vxor_streambuf(stream.rdbuf(), width)) {}

    virtual ~vxor_ostream()
    {
        delete rdbuf();
    }
};


/*** Test main method **********************************************/

int main()
{
    // Read data
    ifstream infile("test.img");
    vxor_istream in(infile, 288);
    char data[144 * 128];
    in.read(data, 144 * 128);
    infile.close();

    // Write data
    ofstream outfile("test2.img");
    vxor_ostream out(outfile, 288);
    out.write(data, 144 * 128);
    out.flush();
    outfile.close();

    return 0;
}

回答by Dietmar Kühl

The proper way to create a new stream in C++ is to derive from std::streambufand to override the underflow()operation for reading and the overflow()and sync()operations for writing. For your purpose you'd create a filtering stream buffer which takes another stream buffer (and possibly a stream from which the stream buffer can be extracted using rdbuf()) as argument and implements its own operations in terms of this stream buffer.

创建C ++中的新流的正确方法是从派生std::streambuf并重写underflow()操作读取和overflow()和sync()操作进行写入。为了您的目的，您将创建一个过滤流缓冲区，它将另一个流缓冲区（可能还有一个可以使用 提取流缓冲区的流rdbuf()）作为参数，并根据此流缓冲区实现自己的操作。

The basic outline of a stream buffer would be something like this:

流缓冲区的基本轮廓是这样的：

class compressbuf
    : public std::streambuf {
    std::streambuf* sbuf_;
    char*           buffer_;
    // context for the compression
public:
    compressbuf(std::streambuf* sbuf)
        : sbuf_(sbuf), buffer_(new char[1024]) {
        // initialize compression context
    }
    ~compressbuf() { delete[] this->buffer_; }
    int underflow() {
        if (this->gptr() == this->egptr()) {
            // decompress data into buffer_, obtaining its own input from
            // this->sbuf_; if necessary resize buffer
            // the next statement assumes "size" characters were produced (if
            // no more characters are available, size == 0.
            this->setg(this->buffer_, this->buffer_, this->buffer_ + size);
        }
        return this->gptr() == this->egptr()
             ? std::char_traits<char>::eof()
             : std::char_traits<char>::to_int_type(*this->gptr());
    }
};

How underflow()looks exactly depends on the compression library being used. Most libraries I have used keep an internal buffer which needs to be filled and which retains the bytes which are not yet consumed. Typically, it is fairly easy to hook the decompression into underflow().

如何underflow()看起来完全取决于所使用的压缩库。我使用的大多数库都有一个内部缓冲区，需要填充它并保留尚未消耗的字节。通常，将解压挂钩到underflow().

Once the stream buffer is created, you can just initialize an std::istreamobject with the stream buffer:

创建流缓冲区后，您可以std::istream使用流缓冲区初始化一个对象：

std::ifstream fin("some.file");
compressbuf   sbuf(fin.rdbuf());
std::istream  in(&sbuf);

If you are going to use the stream buffer frequently, you might want to encapsulate the object construction into a class, e.g., icompressstream. Doing so is a bit tricky because the base class std::iosis a virtual base and is the actual location where the stream buffer is stored. To construct the stream buffer before passing a pointer to a std::iosthus requires jumping through a few hoops: It requires the use of a virtualbase class. Here is how this could look roughly:

如果您打算频繁使用流缓冲区，您可能希望将对象构造封装到一个类中，例如，icompressstream. 这样做有点棘手，因为基类std::ios是一个虚拟基类，并且是存储流缓冲区的实际位置。在将指针传递给 a 之前构造流缓冲区std::ios因此需要跳过几个环节：它需要使用virtual基类。以下是大致的样子：

struct compressstream_base {
    compressbuf sbuf_;
    compressstream_base(std::streambuf* sbuf): sbuf_(sbuf) {}
};
class icompressstream
    : virtual compressstream_base
    , public std::istream {
public:
    icompressstream(std::streambuf* sbuf)
        : compressstream_base(sbuf)
        , std::ios(&this->sbuf_)
        , std::istream(&this->sbuf_) {
    }
};

(I just typed this code without a simple way to test that it is reasonably correct; please expect typos but the overall approach should work as described)

（我只是输入了这段代码，没有用简单的方法来测试它是否正确；请注意错别字，但整体方法应该按描述工作）

回答by Cubbi

boost (which you should have already if you're serious about C++), has a whole library dedicated to extending and customizing IO streams: boost.iostreams

boost（如果你认真对待 C++，你应该已经有了它），有一个专门用于扩展和自定义 IO 流的库：boost.iostreams

In particular, it already has decompressing streams for a few popular formats (bzip2, gzlib, and zlib)

特别是，它已经为一些流行的格式（bzip2、gzlib和zlib）提供了解压缩流

As you saw, extending streambuf may be an involving job, but the library makes it fairly easy to write your own filtering streambufif you need one.

如您所见，扩展 streambuf 可能是一项涉及到的工作，但如果您需要，该库可以很容易地编写您自己的过滤流缓冲区。

回答by Puppy

Don't, unless you want to die a terrible death of hideous design. IOstreams are the worst component of the Standard library - even worse than locales. The iterator model is much more useful, and you can convert from stream to iterator with istream_iterator.

不要，除非你想死于可怕的设计的可怕死亡。IOstreams 是标准库中最糟糕的组件——甚至比语言环境更糟糕。迭代器模型更有用，您可以使用 istream_iterator 将流转换为迭代器。

回答by vitaut

I agree with @DeadMG and wouldn't recommend using iostreams. Apart from poor design the performance is often worse than that of plain old C-style I/O. I wouldn't stick to a particular I/O library though, instead, I'd create an interface (abstract class) that has all required operations, for example:

我同意 @DeadMG 并且不建议使用 iostreams。除了糟糕的设计之外，性能通常比普通的旧 C 风格 I/O 更差。不过，我不会坚持使用特定的 I/O 库，相反，我会创建一个具有所有必需操作的接口（抽象类），例如：

class Input {
 public:
  virtual void read(char *buffer, size_t size) = 0;
  // ...
};

Then you can implement this interface for C I/O, iostreams, mmapor whatever.

然后你可以为 CI/O、iostreamsmmap或其他任何东西实现这个接口。

回答by Nigel Sharp

I think main() given in the Solution in the question has a minor but crucial bug. The ifstream and ofstream should be opened in binary mode:

我认为问题的解决方案中给出的 main() 有一个次要但至关重要的错误。ifstream 和 ofstream 应该以二进制模式打开：

int main()
{
    // Read data
    ifstream infile("test.img", ios::binary);
    ...

    // Write data
    ofstream outfile("test2.img", ios::binary);
    ...
}

Without this I found the reading of the file ended prematurely on Windows

没有这个，我发现文件的读取在 Windows 上过早结束

(I'd have added this as a comment but I don't yet have 50 reputation)

（我已将其添加为评论，但我还没有 50 声望）

回答by Mats Petersson

It is probably possible to do this, but I feel that it's not the "right" usage of this feature in C++. The iostream >> and << operators are meant for fairly simple operations, such as wriitng the "name, street, town, postal code" of a class Person, not for parsing and loading images. That's much better done using the stream::read() - using Image(astream);, and you may implement a stream for compression, as descrtibed by Dietmar.

这样做可能是可能的，但我觉得这不是 C++ 中此功能的“正确”用法。iostream >> 和 << 运算符用于相当简单的操作，例如编写 a 的“名称、街道、城镇、邮政编码” class Person，而不是用于解析和加载图像。使用 stream::read()Image(astream);可以更好地完成 - using ，并且您可以实现用于压缩的流，如 Dietmar 所描述的。