C++ 提升序列化与谷歌协议缓冲区？

boost serialization vs google protocol buffers?

提问by 0xC0DEFACE

Does anyone with experience with these libraries have any comment on which one they preferred? Were there any performance differences or difficulties in using?

有使用过这些库的经验的人对他们更喜欢哪一个有任何评论吗？使用中是否有任何性能差异或困难？

采纳答案by Magnus ?sterlind

I've played around a little with both systems, nothing serious, just some simple hackish stuff, but I felt that there's a real difference in how you're supposed to use the libraries.

我在这两个系统上都玩过一些，没什么大不了的，只是一些简单的hackish 东西，但我觉得你应该如何使用这些库有真正的不同。

With boost::serialization, you write your own structs/classes first, and then add the archiving methods, but you're still left with some pretty "slim" classes, that can be used as data members, inherited, whatever.

使用 boost::serialization，您首先编写自己的结构/类，然后添加归档方法，但您仍然留下一些非常“瘦”的类，它们可以用作数据成员、继承等。

With protocol buffers, the amount of code generated for even a simple structure is pretty substantial, and the structs and code that's generated is more meant for operating on, and that you use protocol buffers' functionality to transport data to and from your own internal structures.

使用协议缓冲区，即使是为简单结构生成的代码量也相当可观，生成的结构和代码更适合用于操作，并且您可以使用协议缓冲区的功能将数据传输到您自己的内部结构和从您自己的内部结构传输数据.

回答by ndfred

I've been using Boost Serialization for a long time and just dug into protocol buffers, and I think they don't have the exact same purpose. BS (didn't see that coming) saves your C++ objects to a stream, whereas PB is an interchange format that you read to/from.

我已经使用 Boost Serialization 很长时间了，只是挖掘了协议缓冲区，我认为它们没有完全相同的目的。BS（没有看到）将您的 C++ 对象保存到流中，而 PB 是您读取/读取的交换格式。

PB's datamodel is way simpler: you get all kinds of ints and floats, strings, arrays, basic structure and that's pretty much it. BS allows you to directly save all of your objects in one step.

PB 的数据模型要简单得多：您可以获得各种整数和浮点数、字符串、数组、基本结构，仅此而已。BS 允许您一步直接保存所有对象。

That means with BS you get more data on the wire but you don't have to rebuild all of your objects structure, whereas protocol buffers is more compact but there is more work to be done after reading the archive. As the name says, one is for protocols (language-agnostic, space efficient data passing), the other is for serialization (no-brainer objects saving).

这意味着使用 BS 您可以在线获得更多数据，但您不必重建所有对象结构，而协议缓冲区更紧凑，但在读取存档后还有更多工作要做。顾名思义，一个用于协议（与语言无关，空间高效的数据传递），另一个用于序列化（无需动脑的对象保存）。

So what is more important to you: speed/space efficiency or clean code?

那么对您来说什么更重要：速度/空间效率还是干净的代码？

回答by bacar

There are a couple of additional concerns with boost.serialization that I'll add to the mix. Caveat: I don't have any direct experience with protocol buffers beyond skimming the docs.

我将添加到 boost.serialization 中还有一些额外的问题。警告：除了浏览文档之外，我对协议缓冲区没有任何直接经验。

Note that while I think boost, and boost.serialization, is great at what it does, I have come to the conclusion that the default archive formats it comes with are not a great choice for a wire format.

请注意，虽然我认为 boost 和 boost.serialization 非常擅长它的功能，但我得出的结论是，它附带的默认存档格式对于有线格式来说并不是一个很好的选择。

It's important to distinguish between versions of your class(as mentioned in other answers, boost.serialization has some support for data versioning) and compatibility between different versions of the serialization library.

区分您的类的版本（如其他答案中所述，boost.serialization 对数据版本控制有一些支持）以及不同版本的序列化库之间的兼容性非常重要。

Newer versions of boost.serialization may not generate archives that older versions can deserialize.(the reverse is not true: newer versions are always intended to deserialize archives made by older versions). This has led to the following problems for us:

较新版本的 boost.serialization可能不会生成旧版本可以反序列化的档案。（反之则不然：新版本总是旨在反序列化旧版本制作的档案）。这给我们带来了以下问题：

• Both our client & server software create serialized objects that the other consumes, so we can only move to a newer boost.serialization if we upgrade both client and server in lockstep. (This is quite a challenge in an environment where you don't have full control of your clients).
• Boost comes bundled as one big library with shared parts, and both the serialization code and the other parts of the boost library (e.g. shared_ptr) may be in use in the same file, I can't upgrade anyparts of boost because I can't upgrade boost.serialization. I'm not sure if it's possible/safe/sane to attempt to link multiple versions of boost into a single executable, or if we have the budget/energy to refactor out bits that need to remain on an older version of boost into a separate executable (DLL in our case).
• The old version of boost we're stuck on doesn't support the latest version of the compiler we use, so we're stuck on an old version of the compiler too.

• 我们的客户端和服务器软件都创建了另一个使用的序列化对象，因此如果我们同步升级客户端和服务器，我们只能转向更新的 boost.serialization。（在您无法完全控制客户的环境中，这是一个相当大的挑战）。
• Boost 捆绑为一个包含共享部分的大库，并且序列化代码和 boost 库的其他部分（例如 shared_ptr）可能在同一个文件中使用，我无法升级boost 的任何部分，因为我可以' t 升级 boost.serialization。我不确定尝试将多个版本的 boost 链接到一个可执行文件中是否可能/安全/理智，或者我们是否有预算/精力将需要保留在旧版本 boost 上的位重构为一个单独的可执行文件（在我们的例子中是 DLL）。
• 我们坚持使用的旧版 boost 不支持我们使用的最新版本的编译器，因此我们也停留在旧版本的编译器上。

Google seem to actually publish the protocol buffers wire format, and Wikipedia describes them as forwards-compatible, backwards-compatible(although I think Wikipedia is referring to data versioning rather than protocol buffer library versioning). Whilst neither of these is a guarantee of forwards-compatibility, it seems like a stronger indication to me.

谷歌似乎实际上发布了协议缓冲区线格式，维基百科将它们描述为向前兼容、向后兼容（尽管我认为维基百科指的是数据版本控制而不是协议缓冲区库版本控制）。虽然这些都不是向前兼容性的保证，但对我来说似乎是一个更有力的迹象。

In summary, I would prefer a well-known, published wire formatlike protocol buffers when I don't have the ability to upgrade client & server in lockstep.

总之，当我无法同步升级客户端和服务器时，我更喜欢众所周知的、已发布的有线格式，如协议缓冲区。

Footnote: shameless plug for a related answerby me.

脚注：我的相关答案的无耻插件。

回答by Nick

Boost Serialisation

提升序列化

• is a library for writing data into a stream.
• does not compress data.
• does not support data versioning automatically.
• supports STL containers.
• properties of data written depend on streams chosen (e.g. endian, compressed).

• 是一个用于将数据写入流的库。
• 不压缩数据。
• 不支持自动数据版本控制。
• 支持 STL 容器。
• 写入数据的属性取决于选择的流（例如字节序、压缩）。

Protocol Buffers

协议缓冲区

• generates code from interface description (supports C++, Python and Java by default. C, C# and others by 3rd party).
• optionally compresses data.
• handles data versioning automatically.
• handles endian swapping between platforms.
• does not support STL containers.

• 从接口描述生成代码（默认支持 C++、Python 和 Java。C、C# 和其他第三方的）。
• 可选地压缩数据。
• 自动处理数据版本控制。
• 处理平台之间的字节序交换。
• 不支持 STL 容器。

Boost serialisation is a library for converting an object into a serialised stream of data. Protocol Buffers do the same thing, but also do other work for you (like versioning and endian swapping). Boost serialisation is simpler for "small simple tasks". Protocol Buffers are probably better for "larger infrastructure".

Boost 序列化是一个用于将对象转换为序列化数据流的库。Protocol Buffers 做同样的事情，但也为你做其他工作（比如版本控制和字节序交换）。对于“小型简单任务”，Boost 序列化更简单。协议缓冲区可能更适合“更大的基础设施”。

EDIT:24-11-10: Added "automatically" to BS versioning.

编辑：24-11-10：将“自动”添加到 BS 版本控制。

回答by Tom

I have no experience with boost serialization, but I have used protocol buffers. I like protocol buffers a lot. Keep the following in mind (I say this with no knowledgeof boost).

我没有提升序列化的经验，但我使用过协议缓冲区。我非常喜欢协议缓冲区。请记住以下几点（我在不了解boost 的情况下这么说）。

• Protocol buffers are very efficient so I don't imaginethat being a serious issue vs. boost.
• Protocol buffers provide an intermediate representation that works with other languages (Python and Java... and more in the works). If you know you're only using C++, maybe boost is better, but the option to use other languages is nice.
• Protocol buffers are more like data containers... there is no object oriented nature, such as inheritance. Think about the structure of what you want to serialize.
• Protocol buffers are flexible because you can add "optional" fields. This basically means you can change the structure of protocol buffer without breaking compatibility.

• 协议缓冲区是非常有效的，所以我不想象这是一个严重的问题与提升。
• 协议缓冲区提供了一种可以与其他语言（Python 和 Java ......以及更多正在开发中的语言）一起使用的中间表示。如果你知道你只使用 C++，也许 boost 更好，但使用其他语言的选项很好。
• 协议缓冲区更像是数据容器......没有面向对象的性质，例如继承。想想你想要序列化的结构。
• 协议缓冲区很灵活，因为您可以添加“可选”字段。这基本上意味着您可以在不破坏兼容性的情况下更改协议缓冲区的结构。

Hope this helps.

希望这可以帮助。

回答by Maik Beckmann

boost.serialization just needs the C++ compiler and gives you some syntax sugar like

boost.serialization 只需要 C++ 编译器并为您提供一些语法糖，例如

serialize_obj >> archive;
// ...
unserialize_obj << archive;

for saving and loading. If C++ is the only language you use you should give boost.serialization a serious shot.

用于保存和加载。如果 C++ 是您使用的唯一语言，您应该认真对待 boost.serialization。

I took a fast look at google protocol buffers. From what I see I'd say its not directly comparable to boost.serialization. You have to add a compiler for the .proto files to your toolchain and maintain the .proto files itself. The API doesn't integrate into C++ as boost.serialization does.

我快速浏览了 google 协议缓冲区。从我所见，我想说它不能直接与 boost.serialization 相提并论。您必须将 .proto 文件的编译器添加到您的工具链中并维护 .proto 文件本身。API 不像 boost.serialization 那样集成到 C++ 中。

boost.serialization does the job its designed for very well: to serialize C++ objects :) OTOH an query-API like google protocol buffers has gives you more flexibility.

boost.serialization 可以很好地完成其设计的工作：序列化 C++ 对象 :) OTOH 像 google protocol buffers 这样的查询 API 为您提供了更大的灵活性。

Since I only used boost.serialization so far I cannot comment on performance comparison.

由于到目前为止我只使用了 boost.serialization，因此我无法评论性能比较。

回答by Robert Ramey

Correction to above (guess this is that answer) about Boost Serialization :

对上面关于 Boost Serialization 的更正（猜想这就是答案）：

It DOES allow supporting data versioning.

它确实允许支持数据版本控制。

If you need compression - use a compressed stream.

如果您需要压缩 - 使用压缩流。

Can handle endian swapping between platforms as encoding can be text, binary or XML.

可以处理平台之间的字节序交换，因为编码可以是文本、二进制或 XML。

回答by i_am_jorf

I never implemented anything using boost's library, but I found Google protobuff's to be more thought-out, and the code is much cleaner and easier to read. I would suggest having a look at the various languages you want to use it with and have a read through the code and the documentation and make up your mind.

我从来没有使用 boost 的库实现过任何东西，但我发现 Google protobuff 的更深思熟虑，代码更清晰，更易于阅读。我建议查看您想要使用它的各种语言，并通读代码和文档并下定决心。

The one difficulty I had with protobufs was they named a very commonly used function in their generated code GetMessage(), which of course conflicts with the Win32 GetMessage macro.

我在使用 protobuf 时遇到的一个困难是他们在生成的代码 GetMessage() 中命名了一个非常常用的函数，这当然与 Win32 GetMessage 宏相冲突。

I would still highly recommend protobufs. They're very useful.

我仍然强烈推荐 protobufs。它们非常有用。

回答by It'sPete

As with almost everything in engineering, my answer is... "it depends."

与工程中的几乎所有事情一样，我的回答是......“这取决于。”

Both are well tested, vetted technologies. Both will take your data and turn it into something friendly for sending someplace. Both will probably be fast enough, and if you're really counting a byte here or there, you're probably not going to be happy with either (let's face it both created packets will be a small fraction of XML or JSON).

两者都是经过充分测试和的技术。两者都将获取您的数据并将其变成友好的东西以发送到某个地方。两者都可能足够快，并且如果您真的在这里或那里计算一个字节，您可能不会对任何一个感到满意（让我们面对这两个创建的数据包都将是 XML 或 JSON 的一小部分）。

For me, it really comes down to workflow and whether or not you need something other than C++ on the other end.

对我来说，这真的归结为工作流程以及在另一端是否需要 C++ 以外的东西。

If you want to figure out your message contents first and you're building a system from scratch, use Protocol Buffers. You can think of the message in an abstract way and then auto-generate the code in whatever language you want (3rd party plugins are available for just about everything). Also, I find collaboration simplified with Protocol Buffers. I just send over a .proto file and then the other team has a clear idea of what data is being transfered. I also don't impose anything on them. If they want to use Java, go ahead!

如果您想首先弄清楚您的消息内容并且您正在从头开始构建一个系统，请使用协议缓冲区。您可以以抽象的方式考虑消息，然后以您想要的任何语言自动生成代码（第 3 方插件几乎适用于所有内容）。此外，我发现使用 Protocol Buffers 简化了协作。我只是发送一个 .proto 文件，然后另一个团队清楚地知道正在传输哪些数据。我也不强加给他们任何东西。如果他们想使用 Java，请继续！

If I already have built a class in C++ (and this has happened more often than not) and I want to send that data over the wire now, Boost Serialization obviously makes a ton of sense (especially where I already have a Boost dependency somewhere else).

如果我已经在 C++ 中构建了一个类（并且这种情况经常发生）并且我现在想通过网络发送该数据，那么 Boost Serialization 显然很有意义（尤其是在我已经在其他地方拥有 Boost 依赖项的情况下） ）。

回答by bazza

I know that this is an older question now, but I thought I'd throw my 2 pence in!

我知道这是一个较旧的问题，但我想我会投入 2 便士！

With boost you get the opportunity to I'm write some data validation in your classes; this is good because the data definition and the checks for validity are all in one place.

随着 boost 你有机会在你的类中编写一些数据验证；这很好，因为数据定义和有效性检查都在一个地方。

With GPB the best you can do is to put comments in the .proto file and hope against all hope that whoever is using it reads it, pays attention to it, and implements the validity checks themselves.

使用 GPB，您能做的最好的事情就是在 .proto 文件中添加注释，并希望使用它的人阅读它、关注它并自行实施有效性检查。

Needless to say this is unlikely and unreliable if your relying on someone else at the other end of a network stream to do this with the same vigour as oneself. Plus if the constraints on validity change, multiple code changes need to be planned, coordinated and done.

毋庸置疑，如果您依靠网络流另一端的其他人以与自己相同的活力来做到这一点，这不太可能也不可靠。此外，如果有效性的约束发生变化，则需要计划、协调和完成多个代码更改。

Thus I consider GPB to be inappropriate for developments where there is little opportunity to regularly meet and talk with all team members.

因此，我认为 GPB 不适用于几乎没有机会定期与所有团队成员会面和交谈的开发项目。

==EDIT==

==编辑==

The kind of thing I mean is this:

我的意思是这样的：

message Foo
{
    int32 bearing = 1;
}

Now who's to say what the valid range of bearingis? We can have

现在谁说有效范围bearing是什么？我们可以有

message Foo
{
    int32 bearing = 1;  // Valid between 0 and 359
}

But that depends on someone else reading this and writing code for it. For example, if you edit it and the constraint becomes:

但这取决于其他人阅读此内容并为其编写代码。例如，如果您对其进行编辑并且约束变为：

message Foo
{
    int32 bearing = 1;  // Valid between -180 and +180
}

you are completely dependent on everyone who has used this .proto updating their code. That is unreliable and expensive.

您完全依赖于使用此 .proto 的每个人更新他们的代码。那是不可靠和昂贵的。

At least with Boost serialisation you're distributing a single C++ class, and that can have data validity checks built right into it. If those constraints change, then no one else need do any work other than making sure they're using the same version of the source code as you.

至少使用 Boost 序列化，您分发的是单个 C++ 类，并且可以在其中内置数据有效性检查。如果这些约束发生变化，那么除了确保他们使用与您相同版本的源代码之外，没有其他人需要做任何工作。

Alternative

选择

There is an alternative: ASN.1. This is ancient, but has some really, really, handy things:

还有一个替代方案：ASN.1。这是古老的，但有一些非常非常方便的东西：

Foo ::= SEQUENCE
{
   bearing INTEGER (0..359)
}

Note the constraint. So whenever anyone consumes this .asn file, generates code, they end up with code that will automatically check that bearingis somewhere between 0 and 359. If you update the .asn file,

注意约束。因此，每当有人使用此 .asn 文件并生成代码时，他们最终都会生成自动检查bearing介于 0 和 359 之间的代码。如果您更新 .asn 文件，

Foo ::= SEQUENCE
{
   bearing INTEGER (-180..180)
}

all they need to do is recompile. No other code changes are required.

他们需要做的就是重新编译。不需要更改其他代码。

You can also do:

你也可以这样做：

bearingMin INTEGER ::= 0
bearingMax INTEGER ::= 360

Foo ::= SEQUENCE
{
   bearing INTEGER (bearingMin..<bearingMax)
}

Note the <. And also in most tools the bearingMin and bearingMax can appear as constants in the generated code. That's extremely useful.

注意<. 而且在大多数工具中，bearingMin 和bearingMax 可以作为常量出现在生成的代码中。这是非常有用的。

Constraints can be quite elaborate:

约束可以非常详细：

Garr ::= INTEGER (0..10 | 25..32)

Look at Chapter 13 in this PDF; it's amazing what you can do;

查看此PDF中的第 13 章；你能做什么真是太棒了；

Arrays can be constrained too:

数组也可以被约束：

Bar ::= SEQUENCE (SIZE(1..5)) OF Foo
Sna ::= SEQUENCE (SIZE(5)) OF Foo
Fee ::= SEQUENCE 
{
    boo SEQUENCE (SIZE(1..<6)) OF INTEGER (-180<..<180)
}

ASN.1 is old fashioned, but still actively developed, widely used (your mobile phone uses it a lot), and far more flexible than most other serialisation technologies. About the only deficiency that I can see is that there is no decent code generator for Python. If you're using C/C++, C#, Java, ADA then you are well served by a mixture of free (C/C++, ADA) and commercial (C/C++, C#, JAVA) tools.

ASN.1 是老式的，但仍在积极开发、广泛使用（您的手机使用它很多），并且比大多数其他序列化技术灵活得多。我能看到的唯一不足是没有像样的 Python 代码生成器。如果您使用的是 C/C++、C#、Java、ADA，那么免费（C/C++、ADA）和商业（C/C++、C#、JAVA）工具的混合可以很好地为您服务。

I especially like the wide choice of binary and text based wireformats. This makes it extremely convenient in some projects. The wireformat list currently includes:

我特别喜欢基于二进制和文本的线格式的广泛选择。这使得它在某些项目中非常方便。线格式列表目前包括：

• BER (binary)
• PER (binary, aligned and unaligned. This is ultra bit efficient. For example, and INTEGER constrained between 0and 15will take up only 4 bitson the wire)
• OER
• DER (another binary)
• XML (also XER)
• JSON (brand new, tool support is still developing)

• BER（二进制）
• PER（二进制，对齐和未对齐。这是非常高效的。例如，INTEGER 约束在0和之间，15将仅占4 bits用在线上）
• 开放式教育资源
• DER（另一个二进制文件）
• XML（也是 XER）
• JSON（全新，工具支持还在开发中）

plus others.

加上其他人。

Note the last two? Yes, you can define data structures in ASN.1, generate code, and emit / consume messages in XML and JSON. Not bad for a technology that started off back in the 1980s.

注意到最后两个了吗？是的，您可以在 ASN.1 中定义数据结构、生成代码以及以 XML 和 JSON 格式发出/使用消息。对于 1980 年代开始的技术来说，这还不错。

Versioning is done differently to GPB. You can allow for extensions:

版本控制与 GPB 不同。您可以允许扩展：

Foo ::= SEQUENCE
{
   bearing INTEGER (-180..180),
   ...
}

This means that at a later date I can add to Foo, and older systems that have this version can still work (but can only access the bearingfield).

这意味着在以后我可以添加到Foo，并且具有此版本的旧系统仍然可以工作（但只能访问该bearing字段）。

I rate ASN.1 very highly. It can be a pain to deal with (tools might cost money, the generated code isn't necessarily beautiful, etc). But the constraints are a truly fantastic feature that has saved me a whole ton of heart ache time and time again. Makes developers whinge a lot when the encoders / decoders report that they've generated duff data.

我对 ASN.1 的评价很高。处理起来可能很痛苦（工具可能要花钱，生成的代码不一定漂亮，等等）。但是约束确实是一个非常棒的功能，它一次又一次地为我节省了大量的心痛。当编码器/解码器报告他们生成了 duff 数据时，让开发人员抱怨很多。

Other links:

其他链接：

• Good intro
• Open source C/C++ compiler
• Open source compiler, does ADA too AFAIK
• Commercial, good
• Commercial, good
• Try it yourself online

• 好介绍
• 开源 C/C++ 编译器
• 开源编译器，ADA 也是 AFAIK
• 商用，不错
• 商用，不错
• 自己上网试试

Observations

观察

To share data:

共享数据：

• Code first approaches (e.g. Boost serialisation) restrict you to the original language (e.g. C++), or force you to do a lot of extra work in another language
• Schema first is better, but
  • A lot of these leave big gaps in the sharing contract (i.e. no constraints). GPB is annoying in this regard, because it is otherwise very good.
  • Some have constraints (e.g. XSD, JSON), but suffer patchy tool support.
  • For example, Microsoft's xsd.exe actively ignores constraints in xsd files (MS's excuse is truly feeble). XSD is good (from the constraints point of view), but if you cannot trust the other guy to use a good XSD tool that enforces them for him/her then the worth of XSD is diminished
  • JSON validators are ok, but they do nothing to help you form the JSON in the first place, and aren't automatically called. There's no guarantee that someone sending you JSON message have run it through a validator. You have to remember to validate it yourself.
  • ASN.1 tools all seem to implement the constraints checking.

• 代码优先的方法（例如 Boost 序列化）将您限制在原始语言（例如 C++）上，或者强迫您使用另一种语言进行大量额外工作
• 模式优先更好，但是
  • 其中很多在共享合同中留下了很大的空白（即没有限制）。GPB 在这方面很烦人，因为它在其他方面非常好。
  • 有些具有约束（例如 XSD、JSON），但受到不完整的工具支持。
  • 例如，微软的 xsd.exe 主动忽略了 xsd 文件中的约束（MS 的借口真的很无力）。XSD 很好（从约束的角度来看），但是如果您不能相信其他人会使用一个好的 XSD 工具来为他/她强制执行，那么 XSD 的价值就会减少
  • JSON 验证器是可以的，但它们首先不会帮助您形成 JSON，并且不会自动调用。无法保证向您发送 JSON 消息的人已通过验证器运行它。你必须记住自己验证它。
  • ASN.1 工具似乎都实现了约束检查。

So for me, ASN.1 does it. It's the one that is least likely to result in someone else making a mistake, because it's the one with the right features and where the tools all seemingly endeavour to fully implement those features, and it is language neutral enough for most purposes.

所以对我来说，ASN.1 做到了。它是最不可能导致其他人犯错误的一个，因为它具有正确的功能，并且工具似乎都在努力完全实现这些功能，并且对于大多数目的来说它是语言中性的。

To be honest, if GPB added a constraints mechanism that'd be the winner. XSD is close but the tools are almost universally rubbish. If there were decent code generators of other languages, JSON schema would be pretty good.

老实说，如果 GPB 添加了约束机制，那将是赢家。XSD 很接近，但工具几乎普遍都是垃圾。如果有像样的其他语言的代码生成器，JSON 模式会非常好。

If GPB had constraints added (note: this would not change any of the wire formats), that'd be the one I'd recommend to everyone for almost every purpose. Though ASN.1's uPER is very useful for radio links.

如果 GPB 添加了约束（注意：这不会改变任何有线格式），那将是我向所有人推荐的几乎所有用途的约束。尽管 ASN.1 的 uPER 对无线电链路非常有用。