从协议缓冲区创建一个类似 Python 字典的对象,以在 Pandas 中使用

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/24682713/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-13 22:14:51  来源:igfitidea点击:

creating a python dictionary like object from protocol buffers for use in pandas

pythonpandasprotocol-buffers

提问by Justin

I currently interface to a server that provides protocol buffers. I can potentially receive a very large number of messages. Currently my process to read the protocol buffers and convert them to a Pandas DataFrame (not a necessary step in general, but Pandas offers nice tools for analyzing datasets) is:

我目前连接到提供协议缓冲区的服务器。我可能会收到大量消息。目前,我读取协议缓冲区并将它们转换为 Pandas DataFrame 的过程(通常不是必要的步骤,但 Pandas 提供了很好的分析数据集的工具)是:

  1. Read protocol buffer, it will be a google protobuf object
  2. Convert protocol buffers to dictionary using protobuf_to_dict
  3. use pandas.DataFrame.from_recordsto get a DataFrame
  1. 读取协议缓冲区,它将是一个 google protobuf 对象
  2. 使用protobuf_to_dict将协议缓冲区转换为字典
  3. 用于pandas.DataFrame.from_records获取 DataFrame

This works great, but, given the large number of messages I read from the protobuf, it is quite inefficient to convert to dictionary and then to pandas. My question is: is it possible to make a class that can make a python protobuf object look like a dictionary? That is, remove step 2. Any references or pseudocode would be helpful.

这很好用,但是,考虑到我从 protobuf 中读取的大量消息,转换为字典然后转换为Pandas的效率非常低。我的问题是:是否有可能制作一个可以使 python protobuf 对象看起来像字典的类?也就是说,删除第 2 步。任何引用或伪代码都会有所帮助。

采纳答案by Zheng Xu

You might want to check the ProtoTextpython package. It does provide in-place dict-like operation to access your protobuf object.

您可能需要检查ProtoTextpython 包。它确实提供了类似 dict 的就地操作来访问您的 protobuf 对象。

Example usage: Assume you have a python protobuf object person_obj.

用法示例:假设您有一个 python protobuf 对象person_obj

import ProtoText
print person_obj['name']       # print out the person_obj.name 
person_obj['name'] = 'David'   # set the attribute 'name' to 'David'
# again set the attribute 'name' to 'David' but in batch mode
person_obj.update({'name': 'David'})
print ('name' in person_obj)  # print whether the 'name' attribute is set in person_obj 
# the 'in' operator is better than the google implementation HasField function 
# in the sense that it won't raise Exception even if the field is not defined