java Netty + ProtoBuffer：一个连接的一些通信消息

Netty + ProtoBuffer: A few communication messages for one connection

提问by Sergii Zagriichuk

While reading the Netty tutorial, I've found a simple descriptionof how to integrate Netty and Google Protocol Buffers. I've started to investigate its example (because there is no more information in the documentation) and written a simple application like the example local time application. But this example is using static initialization in PipeFactory Class, e.g.:

在阅读 Netty 教程时，我发现了一个关于如何集成 Netty 和Google Protocol Buffers的简单描述。我已经开始研究它的示例（因为文档中没有更多信息）并编写了一个简单的应用程序，如示例本地时间应用程序。但是这个例子在 PipeFactory 类中使​​用静态初始化，例如：

import org.jboss.netty.channel.ChannelPipeline;
import org.jboss.netty.channel.ChannelPipelineFactory;
import org.jboss.netty.handler.codec.protobuf.ProtobufDecoder;
import org.jboss.netty.handler.codec.protobuf.ProtobufEncoder;
import org.jboss.netty.handler.codec.protobuf.ProtobufVarint32FrameDecoder;
import org.jboss.netty.handler.codec.protobuf.ProtobufVarint32LengthFieldPrepender;

import static org.jboss.netty.channel.Channels.pipeline;

/**
 * @author sergiizagriichuk
 */
class ProtoCommunicationClientPipeFactory implements ChannelPipelineFactory {

    public ChannelPipeline getPipeline() throws Exception {
        ChannelPipeline p = pipeline();
        p.addLast("frameDecoder", new ProtobufVarint32FrameDecoder());
        p.addLast("protobufDecoder", new ProtobufDecoder(Communication.DataMessage.getDefaultInstance()));

        p.addLast("frameEncoder", new ProtobufVarint32LengthFieldPrepender());
        p.addLast("protobufEncoder", new ProtobufEncoder());

        p.addLast("handler", new ProtoCommunicationClientHandler());
        return p;
    }

}

(Please take a look at line p.addLast("protobufDecoder", new ProtobufDecoder(Communication.DataMessage.getDefaultInstance()));) and just one factory can be created (as I understand) for ClientBootstrapclass, I mean bootstrap.setPipelineFactory()method. So, in this situation I can use ONEmessage to send to server and ONEmessage to receive from server and it is bad for me, and I think not just for me :( How can I use different messages to and from for just one connection? Perhaps I can create a few protobufDecoderlike this

（请查看 line p.addLast("protobufDecoder", new ProtobufDecoder(Communication.DataMessage.getDefaultInstance()));）并且只能为ClientBootstrap类创建一个工厂（据我所知），我的意思是bootstrap.setPipelineFactory()方法。所以，在这种情况下，我可以使用ONE消息发送给服务器，并ONE消息从服务器接收，这是对我不好，我想不只是我:(我如何使用不同的信息，并从其只是一个连接? 也许我可以创建一些protobufDecoder这样的

p.addLast("protobufDecoder", new ProtobufDecoder(Communication.DataMessage.getDefaultInstance()));
p.addLast("protobufDecoder", new ProtobufDecoder(Communication.TestMessage.getDefaultInstance()));
p.addLast("protobufDecoder", new ProtobufDecoder(Communication.SrcMessage.getDefaultInstance()));

or other techniques? Thanks a lot.

或其他技术？非常感谢。

采纳答案by Sergii Zagriichuk

I've found thread of author of netty in google groupsand understood that I have to change my architecture or write my own decoder as I wrote above, So, Start to think what way will be easy and better.

我在google 组中找到了 netty 的作者线程，并且明白我必须像上面写的那样更改我的架构或编写自己的解码器，所以，开始思考什么方法会更容易和更好。

回答by Dominic Cerisano

If you are going to write your own codecs anyway, you might want to look at implementing the Externalizable interface for custom data objects.

如果您无论如何要编写自己的编解码器，您可能需要考虑为自定义数据对象实现 Externalizable 接口。

• Serializable is low-effort, but worst performance (serializes everything).
• Protobuf is a good trade-off between effort and performance (requires .proto maintenance)
• Externalizable is high effort, but best performance (custom minimal codecs)

• Serializable 省力，但性能最差（序列化所有内容）。
• Protobuf 是努力和性能之间的一个很好的权衡（需要 .proto 维护）
• 可外部化的工作量很大，但性能最好（自定义最小编解码器）

If you already know your project will have to scale like a mountain goat, you may have to go the hard road. Protobuf is not a silver bullet.

如果您已经知道您的项目必须像山羊一样扩展，那么您可能不得不走上艰难的道路。Protobuf 不是灵丹妙药。

回答by Arne

the problem is that there is no way to distinct two different protobuf messages from each other in binary format. But there is a way to solve it within the protobuf file:

问题是无法以二进制格式区分两个不同的 protobuf 消息。但是有一种方法可以在 protobuf 文件中解决它：

message AnyMessage {
    message DataMessage { [...] }
    optional DataMessage dataMessage = 1;
    message TestMessage { [...] }
    optional TestMessage testMessage = 2;
    message SrcMessage { [...] }
    optional SrcMessage srcMessage = 3;
}

optional fields that are not set produce no overhead. Additionally you can add an Enum, but it is just a bonus.

未设置的可选字段不会产生开销。此外，您可以添加一个枚举，但这只是一个奖励。

回答by Abe

Theoretically this can be done by modifying the pipeline for each incoming message to suit the incoming message. Take a look at the port unificationexample in Netty.

理论上，这可以通过修改每个传入消息的管道以适应传入消息来完成。看一下Netty中的端口统一示例。

Sequence would be:
1) In frame decoder or another "DecoderMappingDecoder" you check the message type of the incoming message
2) Modify the pipeline dynamically as shown in the example

序列将是：
1）在帧解码器或另一个“DecoderMappingDecoder”中，您检查传入消息的消息类型
2）如示例中所示动态修改管道

But why not use different connections and follow this sequence:
1) Add other decoders in pipeline based on the incoming message only once.
2) Add the sameinstance of channel upstream handler as the last handler in the pipeline, this way all messages get routed to the same instance, which is almost like having a single connection.

但是为什么不使用不同的连接并遵循以下顺序：
1）仅根据传入消息在管道中添加其他解码器一次。
2) 添加与管道中最后一个处理程序相同的通道上游处理程序的实例，这样所有消息都被路由到同一个实例，这几乎就像有一个单一的连接。

回答by devsprint

The issue is not quite a Netty limitation or encoder/decoder limitation. The problem is that Google Protocol Buffers are offering just a way to serialize/deserialize objects, but is not provide a protocol. They have some kind of RPC implementation as part of standard distribution, but if you'll try to implement their RPC protocol then you'll end up with 3 layers of indirection. What I have done in one of the project, was to define a message that is basically an union of messages. This message contains one field that is Type and another field that is the actual message. You'll still end-up with 2 indirection layers, but not 3. In this way the example from Netty will work for you, but as was mention in a previous post, you have to put more logic in the business logic handler.

问题不完全是 Netty 限制或编码器/解码器限制。问题是 Google Protocol Buffers 只提供了一种序列化/反序列化对象的方法，但没有提供协议。他们有某种 RPC 实现作为标准分发的一部分，但是如果您尝试实现他们的 RPC 协议，那么您最终会得到 3 个间接层。我在其中一个项目中所做的是定义一个基本上是消息联合的消息。此消息包含一个字段，即类型，另一个字段是实际消息。您最终仍会得到 2 个间接层，而不是 3 个。通过这种方式，Netty 中的示例将适用于您，但正如在前一篇文章中提到的，您必须在业务逻辑处理程序中放置更多逻辑。

回答by Abhishek kapoor

You can use message tunneling to send various types of messages as payload in a single message. Hope that helps

您可以使用消息隧道将各种类型的消息作为单个消息中的有效负载发送。希望有帮助

回答by Ramanqul Buzaubak

After long research and suffering... I came up with idea of using composition of messages into one wrapper message. Inside that message I use oneofkey to limit the number of allowed objects to theonly one. Checkout the example:

经过长时间的研究和痛苦......我想出了将消息组合成一个包装消息的想法。在该消息中，我使用oneof键将允许的对象数量限制为唯一一个。查看示例：

message OneMessage {
    MessageType messageType = 1;

    oneof messageBody {
        Event event = 2;
        Request request  = 3;
        Response response = 4;
    }

    string messageCode = 5; //unique message code
    int64 timestamp = 6; //server time
}