java Jackson 多个对象和巨大的 json 文件

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/10411020/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-31 00:56:40  来源:igfitidea点击:

Hymanson multiple objects and huge json files

javajsonHymanson

提问by Tom Carrick

I get the feeling that the answer might be a duplicate of this: Hymanson - Json to POJO With Multiple Entriesbut I think that potentially the question is different enough. Also I'm using raw data binding rather than full data binding.

我觉得答案可能与此重复:Hymanson - Json to POJO With Multiple Entries但我认为这个问题可能已经足够不同了。此外,我使用原始数据绑定而不是完整数据绑定。

So like the asker of that question, I have multiple objects in a file and I'm trying to turn them into POJOs and stuff them into a database of my design so I can access the data quickly rather than slowly.

所以就像那个问题的提问者一样,我在一个文件中有多个对象,我试图将它们变成 POJO 并将它们塞进我设计的数据库中,这样我就可以快速而不是缓慢地访问数据。

The files here are in the order of tens of GB, with up to millions of objects in each file. Anyway here is what I have so far:

这里的文件有几十 GB 的数量级,每个文件中最多有数百万个对象。无论如何,这是我到目前为止所拥有的:

ObjectMapper mapper = new ObjectMapper();
Map<String,Object> data = mapper.readValue(new File("foo.json"), Map.class);
System.out.println(data.get("bar"));

And this works great for printing the bar element of the first object in foo, but I need a way to iterate through every element in a way that won't eat up all my memory.

这对于打印 foo 中第一个对象的 bar 元素非常有用,但我需要一种方法来遍历每个元素,而不会占用我所有的内存。

Thanks.

谢谢。

采纳答案by Marko Topolnik

Use this code sample to see the basic idea.

使用此代码示例查看基本思想。

final InputStream in = new FileInputStream("json.json");
try {
  for (Iterator it = new ObjectMapper().readValues(
      new JsonFactory().createJsonParser(in), Map.class); it.hasNext();)
    System.out.println(it.next());
}
finally { in.close();} }

回答by StaxMan

You don't have to choose between Streaming (JsonParser) and ObjectMapper, do both! Traverse a bit with parser, but then call JsonParser.readValueAs(MyType.class)to bind individual JSON Object.

您不必在 Streaming ( JsonParser) 和之间做出选择ObjectMapper,两者都做!用解析器遍历一下,然后调用JsonParser.readValueAs(MyType.class)绑定单个 JSON 对象。

Or, call ObjectMapper's readValue()method passing JsonParser at appropriate points. Or use ObjectMapper.reader(Type.class).readValues()and iterate that way.

或者,在适当的点调用传递 JsonParserObjectMapperreadValue()方法。或者以ObjectMapper.reader(Type.class).readValues()这种方式使用和迭代。

回答by Steven Schlansker

Assuming you have an array wrapping your objects, create a JsonParser and then call readValuesAswith the appropriate type. It gives you back an Iteratorwith all your objects that reads through the file as you consume the objects.

假设您有一个包含对象的数组,请创建一个 JsonParser,然后readValuesAs使用适当的类型进行调用。当您Iterator使用这些对象时,它会为您返回所有读取文件的对象。