Java Kryo 序列化库:是否用于生产?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/2725233/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Kryo serialization library: is it used in production?
提问by Jim Ferrans
Kryois a very new and interesting Java serialization library, and one of the fastest in the thrift-protobufbenchmark. If you've used Kryo, has it already reached enough maturity to try it out in production code?
Kryo是一个非常新颖有趣的 Java 序列化库,也是thrift-protobuf基准测试中速度最快的库之一。如果您使用过 Kryo,它是否已经足够成熟,可以在生产代码中试用它?
Update (10/27/2010):We're using Kryo, though not yet in production. See my answer below for details.
更新 (10/27/2010):我们正在使用 Kryo,但尚未投入生产。有关详细信息,请参阅下面的我的答案。
Update (3/9/2011):Updating to the latest Hymanson and Kryo libraries shows that Hymanson's binary Smile serialization is pretty competitive.
更新 (3/9/2011):更新到最新的 Hymanson 和 Kryo 库表明 Hymanson 的二进制 Smile 序列化非常有竞争力。
采纳答案by NateS
There is a bug reportand a discussion thread. The DateSerializer that comes with Kryo is slightly more efficient size-wise than the SimpleSerializer implementation posted on SO because it uses LongSerializer optimized for positive values.
有一个错误报告和一个讨论线程。Kryo 附带的 DateSerializer 在大小方面比 SO 上发布的 SimpleSerializer 实现更有效,因为它使用针对正值优化的 LongSerializer。
Edit: I forgot to answer the original question. I believe Kryo is used in at least a few production systems. There is mention of it in this article, Jive SBS cache redesign: Part 3. In the Destroy All Humansproject, Kryo is used to communicate with an Android phone that serves as a robot brain (video here).
编辑:我忘了回答原来的问题。我相信 Kryo 至少用于一些生产系统。在这篇文章中提到了它,Jive SBS 缓存重新设计:第 3 部分。在Destroy All Humans项目中,Kryo 用于与充当机器人大脑的 Android 手机进行通信(此处为视频)。
Not a direct answer, but you might browse the Kryo sourceand/or javadocs. Check out the read* and write* methods on the Kryo class, then look at the Serializer class. This is really the core of the library.
不是直接的答案,但您可以浏览Kryo 源代码和/或javadocs。查看 Kryo 类上的 read* 和 write* 方法,然后查看 Serializer 类。这确实是图书馆的核心。
回答by Jim Ferrans
I'll try to answer my own question (Kyro is still very new!).
我会尝试回答我自己的问题(Kyro 还很新!)。
We have a set of about 120 different web services implemented using the Restlet framework. These are consumed by web service clients generally built on top of a Restlet-based client library. The representations sent back and forth between server and client include XML (using the XStream serialization library), JSON (Using Hymanson), XHTML, Java Object Serialization, and as of yesterday, Kryo. So we're in a position to do some solid side-by-side comparisons.
我们有一组使用Restlet 框架实现的大约 120 个不同的 Web 服务。这些由通常构建在基于 Restlet 的客户端库之上的 Web 服务客户端使用。在服务器和客户端之间来回发送的表示包括 XML(使用XStream 序列化库)、JSON(使用Hymanson)、XHTML、Java 对象序列化,以及截至昨天的Kryo。因此,我们可以进行一些可靠的并排比较。
Kryo 1.0.1 seems reasonably stable. Once I actually read up on how to use the API, the only real problem I found was that the default java.util.Date serializer seemed to warp dates a few months into the past. I just had to provide my own override:
Kryo 1.0.1 似乎相当稳定。一旦我真正阅读了如何使用 API,我发现唯一真正的问题是默认的 java.util.Date 序列化程序似乎将日期扭曲到过去几个月。我只需要提供我自己的覆盖:
kryo.register(Date.class,
new SimpleSerializer<Date>() {
@Override public void write (ByteBuffer b, Date d) { b.putLong(d.getTime()); }
@Override public Date read (ByteBuffer b) { return new Date(b.getLong()); }
});
But that was the only possible issue I've found so far. We have a set of JavaBeans that have String, Float, Integer, Long, Date, Boolean and List fields.
但这是我迄今为止发现的唯一可能的问题。我们有一组 JavaBeans,它们有 String、Float、Integer、Long、Date、Boolean 和 List 字段。
Here are some rough benchmarks. First, I did 100,000 serializations and deserializations of an object hierarchy that describes one TV program (ie, made 100,000 deep copies of it). The speeds were:
这里有一些粗略的基准。首先,我对描述一个电视节目的对象层次结构进行了 100,000 次序列化和反序列化(即制作了 100,000 份深度副本)。速度是:
XStream XML: 360/sec
Java Object Serialization: 1,570/sec
Hymanson JSON: 5,000/sec
Kryo: 8,100/sec
Next, I also serialized a catalog of 2,000 TV program descriptions and counted bytes:
接下来,我还连载了一个2000个电视节目描述和字节数的目录:
XStream XML: 6,837,851 bytes
Hymanson JSON: 3,656,654 bytes
Kryo: 1,124,048 bytes
I also found that registering serializers was very important:
我还发现注册序列化程序非常重要:
kryo.register(List.class);
kryo.register(ArrayList.class);
// ...
kryo.register(Program.class);
kryo.register(Catalog.class);
// ...
If I didn't do that, the serializations were almost double the size, and the speed was maybe 40% slower.
如果我不这样做,序列化的大小几乎是原来的两倍,速度可能会慢 40%。
We also ran complete end-to-end tests of several web services using each of these four serialization methods, and they also showed that Kryo was running faster than the others.
我们还使用这四种序列化方法中的每一种对多个 Web 服务进行了完整的端到端测试,结果还表明 Kryo 的运行速度比其他方法快。
So in summary, Kryo seems reasonably robust. I'm going to keep support for it in our code base and as we gain experience with it I hope to use it in more places. Kudos to the Kryo team!
总而言之,Kryo 似乎相当稳健。我将在我们的代码库中继续支持它,随着我们获得使用它的经验,我希望在更多地方使用它。向 Kryo 团队致敬!
Update (3/9/2011):I finally got around to @StaxMan's suggestion to try Hymanson 1.6's binary "Smile" serializer. Using Hymanson 1.6 and Kryo 1.04, I did 100,000 deep copies (serialization/deserialiations) of a somewhat different TV program object hierarchy:
更新(2011 年 3 月 9 日):我终于接受了@StaxMan 的建议,尝试使用 Hymanson 1.6 的二进制“Smile”序列化程序。使用 Hymanson 1.6 和 Kryo 1.04,我做了 100,000 次深度拷贝(序列化/反序列化)的有点不同的电视节目对象层次结构:
XStream XML: 429/sec 5,189 bytes
Hymanson JSON: 4,474/sec 2,657 bytes
Kryo: 4,539/sec 1,066 bytes
Hymanson Smile: 5,040/sec 1,689 bytes
This test didn't mesh with a macro-level test, where I tried different serializers in a REST web service that delivers many of these objects. There the overall system throughput supports @StaxMan's intuition about performance:
该测试与宏观测试不匹配,我在提供许多这些对象的 REST Web 服务中尝试了不同的序列化程序。整体系统吞吐量支持@StaxMan 关于性能的直觉:
Hymanson JSON: 92 requests/sec
Hymanson Smile 97 requests/sec
Kryo: 108 requests/sec
回答by Sanmi
Kryo is part of Yahoo's S4 (Simple Scalable Streaming System) project. S4 isn't production yet as far as I know.
Kryo 是雅虎 S4(Simple Scalable Streaming System)项目的一部分。据我所知,S4 还没有量产。
回答by Chris Dennett
The latest version of Kryo has a few race conditions in some extreme cases, running on a simulator interface to ns-3 from Java. Might ask the developer to commit some of my changes back if they are problem free.
最新版本的 Kryo 在一些极端情况下有一些竞争条件,运行在来自 Java 的 ns-3 的模拟器接口上。如果他们没有问题,可能会要求开发人员提交我的一些更改。
回答by Jagoliveira
With the help of Jim Ferransresponses and comments above I found a more detailed explanation about Date Serialization Issue with Kryo on this page: http://groups.google.com/group/kryo-users/browse_thread/thread/91969c6f48a45bdf/and also a how to use DateSerializer() of Kryo:
在上述Jim Ferrans回复和评论的帮助下,我在此页面上找到了有关 Kryo 的日期序列化问题的更详细说明:http: //groups.google.com/group/kryo-users/browse_thread/thread/91969c6f48a45bdf/以及Kryo 的 DateSerializer() 的使用方法:
kryo.register(Date.class, new DateSerializer());
kryo.register(Date.class, new DateSerializer());
I hope this could help others.
我希望这可以帮助其他人。
回答by RAbraham
The Kryo site has section on projects in production using Kryo
Kryo 网站上有关于使用 Kryo 进行生产的项目的部分
回答by Richard Burkhardt
Apache Storm uses it for serializationbefore passing messages from one task to another.
Apache Storm在将消息从一个任务传递到另一个任务之前使用它进行序列化。
So yes it must be quite stable since Storm is used by several huge companies, i.e., Twitter and Spotify.
所以是的,它必须非常稳定,因为 Storm 被几家大公司使用,即 Twitter 和 Spotify。
回答by Hymanrabb1t
Kryo 2.x is also used by Mule ESB, and so widely used in production.
Kryo 2.x 也被 Mule ESB 使用,因此在生产中被广泛使用。
回答by Ajeet Ganga
2017 update:
2017年更新:
Kryo is used by Flink. So practically anything that is using Flink framework is relying on Kryo. Reference: https://ci.apache.org/projects/flink/flink-docs-release-0.8/programming_guide.html#specifying-keys
Flink 使用 Kryo。所以几乎任何使用 Flink 框架的东西都依赖于 Kryo。参考:https: //ci.apache.org/projects/flink/flink-docs-release-0.8/programming_guide.html#specifying-keys