java 在 Android 上使用 Jackson 库解析大型 JSON 时出现内存不足错误

Out of memory error while parsing a large JSON using Hymanson library on Android

提问by Ramit

I am using Hymanson library to parse a large JSON response from server. Size of the json is around 7-8 mb.

我正在使用 Hymanson 库来解析来自服务器的大型 JSON 响应。json 的大小约为 7-8 mb。

I am getting the outOfMemoryError on this piece of code:

我在这段代码上收到 outOfMemoryError 错误：

ObjectMapper mapper = new ObjectMapper();
JsonNode rootParser = mapper.readValue(is, JsonNode.class);

and this is the exception that I am getting:

这是我得到的例外：

    01-14 13:13:20.103: E/AndroidRuntime(25468): FATAL EXCEPTION: Thread-13
    01-14 13:13:20.103: E/AndroidRuntime(25468): java.lang.OutOfMemoryError
    01-14 13:13:20.103: E/AndroidRuntime(25468): at java.util.ArrayList.add(ArrayList.java:123)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.node.ArrayNode._add(ArrayNode.java:722)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.node.ArrayNode.add(ArrayNode.java:203)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at org.codehaus.Hymanson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at com.sarla.smartglance.communication.JsonDecoder.decodeResponse(JsonDecoder.java:87)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at com.sarla.smartglance.communication.JsonDecoder.decode(JsonDecoder.java:68)
01-14 13:13:20.103: E/AndroidRuntime(25468):    at com.sarla.smartglance.communication.MHttpManager.run(MHttpManager.java:86)

I tried everything but couldn't find any solution to parse such a large amount of data on android.

我尝试了所有方法，但找不到任何解决方案来解析 android 上如此大量的数据。

回答by StaxMan

With 7-8 megs of JSON, tree model that you are using will typically use 20 - 50 megs of memory (dom models are 3-5x as big, both for XML and JSON). There isn't much you can do about that, regardless of library used: they all build trees using Lists and Maps, which is a heavy-weight way of doing it.

对于 7-8 兆的 JSON，您使用的树模型通常会使用 20-50 兆的内存（DOM 模型是 3-5 倍，对于 XML 和 JSON）。无论使用什么库，您都无能为力：它们都使用Lists 和Maps构建树，这是一种重量级的方法。

Instead you should consider using Plain Old Java Objects (POJOs), which will use much less memory. For this you need to model POJO that matches your JSON structure; without knowing structure I can't give an example (if you add sample on question, I can), but code to parse is similar to GSON code referenced by another answer:

相反，您应该考虑使用普通旧 Java 对象 (POJO)，这将使用更少的内存。为此，您需要对与 JSON 结构匹配的 POJO 进行建模；在不知道结构的情况下，我无法举例（如果您在问题上添加示例，我可以），但要解析的代码类似于另一个答案引用的 GSON 代码：

MyValue value = mapper.readValue(json, MyValue.class);

This will work on Hymanson as well as many other Java JSON libs (Gson, Genson at least), and will also be faster method to use. JSON trees are inherently expensive and heavy-weight, and not to be used for multi-megatbyte content.

这将适用于 Hymanson 以及许多其他 Java JSON 库（至少是 Gson、Genson），并且使用起来也会更快。JSON 树本质上是昂贵且重量级的，不能用于多兆字节的内容。

Finally, if your input consists of a sequence of items, there are even better ways to slice it up (which can be done regardless of whether individual items would be JsonNodes or POJOs!). But I don't know if your content is like that.

最后，如果您的输入由一系列项目组成，则有更好的方法将其切片（无论单个项目是JsonNodes 还是 POJO ，都可以这样做！）。但我不知道你的内容是不是这样。

回答by William Da Silva

We use here the gson lib, and with the code above we can get files larger than 50Mb without problem:

我们在这里使用 gson 库，通过上面的代码，我们可以毫无问题地获得大于 50Mb 的文件：

public static <T extends Object> T readFile(String caminho_arquivo, Type type) {

    GsonBuilder gson_builder = new GsonBuilder();

    final SimpleDateFormat sdf_date     = new SimpleDateFormat("yyyy-MM-dd");
    final SimpleDateFormat sdf_datetime = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");

    gson_builder.registerTypeAdapter(Date.class, new JsonDeserializer<Date>(){

        @Override
        public Date deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {

            try {
                if (json.getAsJsonPrimitive().getAsString().length() == 10)
                    return sdf_date.parse(json.getAsJsonPrimitive().getAsString());
                else
                    return sdf_datetime.parse(json.getAsJsonPrimitive().getAsString());

            } catch (ParseException e) {
                Log.e("JSON", "Erro na deserializa??o de datas no JSON: " + json.getAsJsonPrimitive().getAsString());
                return null;
            }
        }

    });

    Gson gson = gson_builder.create();

    File fileJSON = new File(caminho_arquivo);

    FileReader reader = null;

    try {
        reader = new FileReader(fileJSON);

        return gson.fromJson(reader, type);

    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } finally {
        try {
            reader.close();

            if (fileJSON.exists())
                fileJSON.delete();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    return null;
}

Try this lib, that's a good one, with the Hymanson we use only in the server side, because Hymanson is more slow in the Android than gson, at least in our test.

试试这个库，这是一个很好的库，我们只在服务器端使用 Hymanson，因为 Hymanson 在 Android 中比 gson 慢，至少在我们的测试中是这样。

回答by fge

Try and use:

尝试使用：

JsonNode rootParser = mapper.readTree(is);

instead.

反而。