java java中PB的通用解析

Generic Parsing of PB in java

提问by Jimm

Is it possible to parse protobuf in a generic fashion in Java?

是否可以在 Java 中以通用方式解析 protobuf？

I have looked into GeneratedMessage and could not find a way to parse any PB byte buffer into a GeneratedMessage.

我查看了 GeneratedMessage，但找不到将任何 PB 字节缓冲区解析为 GeneratedMessage 的方法。

Essentially, I am trying to parse a PB byte buffer into GeneratedMessage and then I would use reflection to detect fields inside it.

本质上，我试图将 PB 字节缓冲区解析为 GeneratedMessage，然后我将使用反射来检测其中的字段。

回答by Kannan Goundan

First of all, you can't parse PB data without knowing the schema. The schema originally comes from a ".proto" file and is typically embedded in the code generated by protoc. However, you can also tell protocto store the schema in a format that's usable by the Java Protobuf library:

首先，不知道schema是无法解析PB数据的。该架构最初来自“.proto”文件，通常嵌入在由 .proto 生成的代码中protoc。但是，您也可以告诉protoc以 Java Protobuf 库可用的格式存储模式：

protoc --descriptor_set_out=mymessages.desc mymessages.proto

Then load it in your Java code:

然后将其加载到您的 Java 代码中：

FileInputStream fin = new FileInputStream("mymessages.desc");
Descriptors.FileDescriptorSet set =
  Descriptors.FileDescriptorSet.parseFrom(fin);
Descriptors.Descriptor md = set.getFile(0).getMessageType(0);

Once you have the schema for a message (Descriptor.Descriptor) parsing a message is easy:

一旦您拥有消息 ( Descriptor.Descriptor)的架构，解析消息就很容易了：

byte[] data = ...;
DynamicMessage m = DynamicMessage.parseFrom(md, data);

DynamicMessagehas a reflective API that lets you look through the fields.

DynamicMessage有一个反射 API，可让您查看字段。

The messy part is calling out to the protoctool to convert the ".proto" file into a usable format. The C++ Protobuf library has a way to load ".proto" files directly, but unfortunately the Java Protobuf library does not.

麻烦的部分是调用该protoc工具将“.proto”文件转换为可用格式。C++ Protobuf 库有一种直接加载“.proto”文件的方法，但不幸的是 Java Protobuf 库没有。

回答by Rib47

I have working solution tested with last protobuf v.3.1.0

我已经使用最后一个 protobuf v.3.1.0 测试了工作解决方案

This is upgraded solution raised on previous answers. Thanks to both authors.

这是在以前的答案中提出的升级解决方案。感谢两位作者。

import com.example.address.AddressBookManager;
import com.example.address.AddressBookProtos.AddressBook;
import com.google.protobuf.DescriptorProtos.FileDescriptorSet;
import com.google.protobuf.Descriptors;
import com.google.protobuf.Descriptors.Descriptor;
import com.google.protobuf.Descriptors.FileDescriptor;
import com.google.protobuf.DynamicMessage;

import java.io.File;
import java.io.InputStream;

public class DynamicMessageDemo {

    private static final String ADDRESS_BOOK_SOURCE_FILENAME = "test.ab";
    private static final String ADDRESS_BOOK_DESC_FILENAME =
        File.separator + "address.desc";

    public static void main(String[] args) throws Exception {

        InputStream is = DynamicMessageDemo.class
                .getResourceAsStream(ADDRESS_BOOK_DESC_FILENAME);

        FileDescriptorSet set = FileDescriptorSet.parseFrom(is);
        FileDescriptor fd = Descriptors.FileDescriptor.buildFrom(
                set.getFile(0), 
                new Descriptors.FileDescriptor[]{}
                );

        // "AddressBook" is the second message in my *.proto
        // so index must be '1'
        Descriptor messageType = fd.getMessageTypes().get(1);

        // for testing purpose 
        AddressBook book = AddressBookManager.readFromFile(ADDRESS_BOOK_SOURCE_FILENAME);
        byte[] data = book.toByteArray();

        DynamicMessage message = DynamicMessage.parseFrom(messageType, data);

        System.out.println("\n Dynamic message:\n" + message);
    }
}

回答by vlp

You can use UnknownFieldSetto parse generic protobuf messages.

您可以使用UnknownFieldSet来解析通用 protobuf 消息。

Then you can get individual fields using provided methods (e.g. asMap(), hasField(), getField())

然后你可以使用提供的方法获取单个字段（例如asMap()、hasField()、getField()）

E.g. (data taken from this question):

例如（取自这个问题的数据）：

    byte[] msg = new byte[] { 0x0a, 0x10, 0x08, 0x7f, (byte)0x8a, 0x01, 0x04, 0x08, 0x02, 0x10, 0x03, (byte)0x92, 0x01, 0x04, 0x08, 0x02, 0x10, 0x03, 0x18, 0x01};
    UnknownFieldSet eee = UnknownFieldSet.parseFrom(msg);
    System.out.println(eee.toString());

Giving:

给予：

1: {
  1: 127
  17: {
    1: 2
    2: 3
  }
  18: {
    1: 2
    2: 3
  }
}
3: 1

回答by Бешеный Пингвин

This is right example:

这是正确的例子：

private static DynamicMessage parseData(byte[] data) throws IOException, DescriptorValidationException {
    FileInputStream fin = new FileInputStream("test.desc");
    DescriptorProtos.FileDescriptorSet set = DescriptorProtos.FileDescriptorSet.parseFrom(fin);
    Descriptor md = Descriptors.FileDescriptor.buildFrom(set.getFile(0), new  Descriptors.FileDescriptor[] {}).findMessageTypeByName("Person");
    return DynamicMessage.parseFrom(md, data);
}

回答by pepe1

Here is another way to generically parse a .desc file:

这是一般解析 .desc 文件的另一种方法：

FileInputStream fin = new FileInputStream(descPath);
DescriptorProtos.FileDescriptorSet set = DescriptorProtos.FileDescriptorSet.parseFrom(fin);

List<FileDescriptor> dependencyFileDescriptorList = new ArrayList<>();
for(int i=0; i<set.getFileCount()-1;i++) {
    dependencyFileDescriptorList.add(Descriptors.FileDescriptor.buildFrom(set.getFile(i), new  Descriptors.FileDescriptor[] {}));
}
FileDescriptor desc = Descriptors.FileDescriptor.buildFrom(set.getFile(set.getFileCount()-1), dependencyFileDescriptorList.toArray(new FileDescriptor[0]));