C++ 如何使用 zlib 压缩缓冲区？

How to compress a buffer with zlib?

提问by Richard Knop

There is a usage example at the zlib website: http://www.zlib.net/zlib_how.html

zlib 网站上有一个用法示例：http: //www.zlib.net/zlib_how.html

However in the example they are compressing a file. I would like to compress a binary data stored in a buffer in memory. I don't want to save the compressed buffer to disk either.

但是在示例中，他们正在压缩文件。我想压缩存储在内存缓冲区中的二进制数据。我也不想将压缩的缓冲区保存到磁盘。

Basically here is my buffer:

基本上这是我的缓冲区：

fIplImageHeader->imageData = (char*)imageIn->getFrame();

How can I compress it with zlib?

如何使用 zlib 压缩它？

I would appreciate some code example of how to do that.

我会很感激一些如何做到这一点的代码示例。

采纳答案by Jonas Gulle

This is an example to pack a buffer with zlib and save the compressed contents in a vector.

这是一个使用 zlib 打包缓冲区并将压缩内容保存在向量中的示例。

void compress_memory(void *in_data, size_t in_data_size, std::vector<uint8_t> &out_data)
{
 std::vector<uint8_t> buffer;

 const size_t BUFSIZE = 128 * 1024;
 uint8_t temp_buffer[BUFSIZE];

 z_stream strm;
 strm.zalloc = 0;
 strm.zfree = 0;
 strm.next_in = reinterpret_cast<uint8_t *>(in_data);
 strm.avail_in = in_data_size;
 strm.next_out = temp_buffer;
 strm.avail_out = BUFSIZE;

 deflateInit(&strm, Z_BEST_COMPRESSION);

 while (strm.avail_in != 0)
 {
  int res = deflate(&strm, Z_NO_FLUSH);
  assert(res == Z_OK);
  if (strm.avail_out == 0)
  {
   buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
   strm.next_out = temp_buffer;
   strm.avail_out = BUFSIZE;
  }
 }

 int deflate_res = Z_OK;
 while (deflate_res == Z_OK)
 {
  if (strm.avail_out == 0)
  {
   buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE);
   strm.next_out = temp_buffer;
   strm.avail_out = BUFSIZE;
  }
  deflate_res = deflate(&strm, Z_FINISH);
 }

 assert(deflate_res == Z_STREAM_END);
 buffer.insert(buffer.end(), temp_buffer, temp_buffer + BUFSIZE - strm.avail_out);
 deflateEnd(&strm);

 out_data.swap(buffer);
}

回答by Huiwei

zlib.hhas all the functions you need: compress(or compress2) and uncompress. See the source code of zlib for an answer.

zlib.h拥有您需要的所有功能：（compress或compress2）和uncompress。请参阅 zlib 的源代码以获得答案。

ZEXTERN int ZEXPORT compress OF((Bytef *dest,   uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
         Compresses the source buffer into the destination buffer.  sourceLen is
     the byte length of the source buffer.  Upon entry, destLen is the total size
     of the destination buffer, which must be at least the value returned by
     compressBound(sourceLen).  Upon exit, destLen is the actual size of the
     compressed buffer.

         compress returns Z_OK if success, Z_MEM_ERROR if there was not
     enough memory, Z_BUF_ERROR if there was not enough room in the output
     buffer.
*/

ZEXTERN int ZEXPORT uncompress OF((Bytef *dest,   uLongf *destLen, const Bytef *source, uLong sourceLen));
/*
         Decompresses the source buffer into the destination buffer.  sourceLen is
     the byte length of the source buffer.  Upon entry, destLen is the total size
     of the destination buffer, which must be large enough to hold the entire
     uncompressed data.  (The size of the uncompressed data must have been saved
     previously by the compressor and transmitted to the decompressor by some
     mechanism outside the scope of this compression library.) Upon exit, destLen
     is the actual size of the uncompressed buffer.

         uncompress returns Z_OK if success, Z_MEM_ERROR if there was not
     enough memory, Z_BUF_ERROR if there was not enough room in the output
     buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete.  In
     the case where there is not enough room, uncompress() will fill the output
     buffer with the uncompressed data up to that point.
*/

回答by kichik

You can easily adapt the example by replacing fread()and fwrite()calls with direct pointers to your data. For zlib compression (referred to as deflateas you "take out all the air of your data") you allocate z_streamstructure, call deflateInit()and then:

您可以通过使用指向数据的直接指针替换fread()和fwrite()调用来轻松调整示例。对于 zlib 压缩（称为deflate，因为您“取出数据的所有空气”），您分配z_stream结构，调用deflateInit()，然后：

1. fill next_inwith the next chunk of data you want to compress
2. set avail_into the number of bytes available in next_in
3. set next_outto where the compressed data should be written which should usually be a pointer inside your buffer that advances as you go along
4. set avail_outto the number of bytes available in next_out
5. call deflate
6. repeat steps 3-5 until avail_outis non-zero (i.e. there's more room in the output buffer than zlib needs - no more data to write)
7. repeat steps 1-6 while you have data to compress

1. 填充next_in下一个要压缩的数据块
2. 设置avail_in为可用字节数next_in
3. 设置next_out为应该写入压缩数据的位置，通常应该是缓冲区内的指针，随着您的进行而前进
4. 设置avail_out为可用字节数next_out
5. 称呼 deflate
6. 重复步骤 3-5 直到avail_out不为零（即输出缓冲区中有比 zlib 需要的更多空间 - 没有更多数据要写入）
7. 当您有数据要压缩时，重复步骤 1-6

Eventually you call deflateEnd()and you're done.

最后你打电话deflateEnd()，你就完成了。

You're basically feeding it chunks of input and output until you're out of input and it is out of output.

你基本上是在给它提供大量的输入和输出，直到你没有输入并且没有输出。

回答by Yippie-Ki-Yay

This is not a direct answer on your question about the zlib API, but you may be interested in boost::iostreamslibrary paired with zlib.

这不是您有关 zlib API 问题的直接答案，但您可能boost::iostreams对与zlib.

This allows to use zlib-driven packing algorithms using the basic "stream" operations notation and then your data could be easily compressed by opening some memory stream and doing the << dataoperation on it.

这允许使用zlib使用基本“流”操作符号的驱动打包算法，然后您的数据可以通过打开一些内存流并对其进行<< data操作来轻松压缩。

In case of boost::iostreamsthis would automatically invoke the corresponding packing filter for every data that passes through the stream.

在boost::iostreams这种情况下，将为通过流的每个数据自动调用相应的打包过滤器。

回答by BullyWiiPlaza

The classic way more convenient with C++ features

使用 C++ 特性更方便的经典方式

Here's a full example which demonstrates compression and decompression using C++std::vectorobjects:

这是一个完整的示例，它演示了使用C++std::vector对象进行压缩和解压缩：

#include <cstdio>
#include <iosfwd>
#include <iostream>
#include <vector>
#include <zconf.h>
#include <zlib.h>
#include <iomanip>
#include <cassert>

void add_buffer_to_vector(std::vector<char> &vector, const char *buffer, uLongf length) {
    for (int character_index = 0; character_index < length; character_index++) {
        char current_character = buffer[character_index];
        vector.push_back(current_character);
    }
}

int compress_vector(std::vector<char> source, std::vector<char> &destination) {
    unsigned long source_length = source.size();
    uLongf destination_length = compressBound(source_length);

    char *destination_data = (char *) malloc(destination_length);
    if (destination_data == nullptr) {
        return Z_MEM_ERROR;
    }

    Bytef *source_data = (Bytef *) source.data();
    int return_value = compress2((Bytef *) destination_data, &destination_length, source_data, source_length,
                                 Z_BEST_COMPRESSION);
    add_buffer_to_vector(destination, destination_data, destination_length);
    free(destination_data);
    return return_value;
}

int decompress_vector(std::vector<char> source, std::vector<char> &destination) {
    unsigned long source_length = source.size();
    uLongf destination_length = compressBound(source_length);

    char *destination_data = (char *) malloc(destination_length);
    if (destination_data == nullptr) {
        return Z_MEM_ERROR;
    }

    Bytef *source_data = (Bytef *) source.data();
    int return_value = uncompress((Bytef *) destination_data, &destination_length, source_data, source.size());
    add_buffer_to_vector(destination, destination_data, destination_length);
    free(destination_data);
    return return_value;
}

void add_string_to_vector(std::vector<char> &uncompressed_data,
                          const char *my_string) {
    int character_index = 0;
    while (true) {
        char current_character = my_string[character_index];
        uncompressed_data.push_back(current_character);

        if (current_character == '
int main(int argc, char *argv[]) {
    test_compression();
    return EXIT_SUCCESS;
}
') {
            break;
        }

        character_index++;
    }
}

// https://stackoverflow.com/a/27173017/3764804
void print_bytes(std::ostream &stream, const unsigned char *data, size_t data_length, bool format = true) {
    stream << std::setfill('0');
    for (size_t data_index = 0; data_index < data_length; ++data_index) {
        stream << std::hex << std::setw(2) << (int) data[data_index];
        if (format) {
            stream << (((data_index + 1) % 16 == 0) ? "\n" : " ");
        }
    }
    stream << std::endl;
}

void test_compression() {
    std::vector<char> uncompressed(0);
    auto *my_string = (char *) "Hello, world!";
    add_string_to_vector(uncompressed, my_string);

    std::vector<char> compressed(0);
    int compression_result = compress_vector(uncompressed, compressed);
    assert(compression_result == F_OK);

    std::vector<char> decompressed(0);
    int decompression_result = decompress_vector(compressed, decompressed);
    assert(decompression_result == F_OK);

    printf("Uncompressed: %s\n", uncompressed.data());
    printf("Compressed: ");
    std::ostream &standard_output = std::cout;
    print_bytes(standard_output, (const unsigned char *) compressed.data(), compressed.size(), false);
    printf("Decompressed: %s\n", decompressed.data());
}

In your main.cppsimply call:

在您的main.cpp简单调用中：

Uncompressed: Hello, world!
Compressed: 78daf348cdc9c9d75128cf2fca495164000024e8048a
Decompressed: Hello, world!

The output produced:

产生的输出：

#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>

std::string compress(const std::string &data) {
    boost::iostreams::filtering_streambuf<boost::iostreams::output> output_stream;
    output_stream.push(boost::iostreams::zlib_compressor());
    std::stringstream string_stream;
    output_stream.push(string_stream);
    boost::iostreams::copy(boost::iostreams::basic_array_source<char>(data.c_str(),
                                                                      data.size()), output_stream);
    return string_stream.str();
}

std::string decompress(const std::string &cipher_text) {
    std::stringstream string_stream;
    string_stream << cipher_text;
    boost::iostreams::filtering_streambuf<boost::iostreams::input> input_stream;
    input_stream.push(boost::iostreams::zlib_decompressor());

    input_stream.push(string_stream);
    std::stringstream unpacked_text;
    boost::iostreams::copy(input_stream, unpacked_text);
    return unpacked_text.str();
}

TEST_CASE("zlib") {
    std::string plain_text = "Hello, world!";
    const auto cipher_text = compress(plain_text);
    const auto decompressed_plain_text = decompress(cipher_text);
    REQUIRE(plain_text == decompressed_plain_text);
}

The Boost way

提升方式

##代码##