Json 字符串到 Java 对象 Avro
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/27559543/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Json String to Java Object Avro
提问by Princey James
I am trying to convert a Json string into a generic Java Object, with an Avro Schema.
我正在尝试将 Json 字符串转换为具有 Avro 架构的通用 Java 对象。
Below is my code.
下面是我的代码。
String json = "{\"foo\": 30.1, \"bar\": 60.2}";
String schemaLines = "{\"type\":\"record\",\"name\":\"FooBar\",\"namespace\":\"com.foo.bar\",\"fields\":[{\"name\":\"foo\",\"type\":[\"null\",\"double\"],\"default\":null},{\"name\":\"bar\",\"type\":[\"null\",\"double\"],\"default\":null}]}";
InputStream input = new ByteArrayInputStream(json.getBytes());
DataInputStream din = new DataInputStream(input);
Schema schema = Schema.parse(schemaLines);
Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
DatumReader<Object> reader = new GenericDatumReader<Object>(schema);
Object datum = reader.read(null, decoder);
I get "org.apache.avro.AvroTypeException: Expected start-union. Got VALUE_NUMBER_FLOAT" Exception.
我得到“org.apache.avro.AvroTypeException: Expected start-union. Got VALUE_NUMBER_FLOAT”异常。
The same code works, if I don't have unions in the schema. Can someone please explain and give me a solution.
如果模式中没有联合，则相同的代码有效。有人可以解释一下并给我一个解决方案。
采纳答案by miljanm
Your schema does not match the schema of the json string. You need to have a different schema that does not have a union in the place of the error but a decimal number. Such schema should then be used as a writer schema while you can freely use the other one as the reader schema.
您的架构与 json 字符串的架构不匹配。您需要有一个不同的架构，该架构在错误位置没有联合，而是一个十进制数。这样的模式应该被用作编写器模式，而您可以自由地使用另一个作为读取器模式。
回答by Liang
Thanks to Reza. I found this webpage. It introduces how to convert a Json string into an avro object.
感谢雷扎。我找到了这个网页。它介绍了如何将 Json 字符串转换为 avro 对象。
http://rezarahim.blogspot.com/2013/06/import-org_26.html
http://rezarahim.blogspot.com/2013/06/import-org_26.html
The key of his code is:
他的代码的关键是：
static byte[] fromJsonToAvro(String json, String schemastr) throws Exception {
InputStream input = new ByteArrayInputStream(json.getBytes());
DataInputStream din = new DataInputStream(input);
Schema schema = Schema.parse(schemastr);
Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
DatumReader<Object> reader = new GenericDatumReader<Object>(schema);
Object datum = reader.read(null, decoder);
GenericDatumWriter<Object> w = new GenericDatumWriter<Object>(schema);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
Encoder e = EncoderFactory.get().binaryEncoder(outputStream, null);
w.write(datum, e);
e.flush();
return outputStream.toByteArray();
}
String json = "{\"username\":\"miguno\",\"tweet\":\"Rock: Nerf paper, scissors is fine.\",\"timestamp\": 1366150681 }";
String schemastr ="{ \"type\" : \"record\", \"name\" : \"twitter_schema\", \"namespace\" : \"com.miguno.avro\", \"fields\" : [ { \"name\" : \"username\", \"type\" : \"string\", \"doc\" : \"Name of the user account on Twitter.com\" }, { \"name\" : \"tweet\", \"type\" : \"string\", \"doc\" : \"The content of the user's Twitter message\" }, { \"name\" : \"timestamp\", \"type\" : \"long\", \"doc\" : \"Unix epoch time in seconds\" } ], \"doc:\" : \"A basic schema for storing Twitter messages\" }";
byte[] avroByteArray = fromJsonToAvro(json,schemastr);
Schema schema = Schema.parse(schemastr);
DatumReader<Genericrecord> reader1 = new GenericDatumReader<Genericrecord>(schema);
Decoder decoder1 = DecoderFactory.get().binaryDecoder(avroByteArray, null);
GenericRecord result = reader1.read(null, decoder1);
回答by Valloric
With Avro 1.4.1, this works:
使用 Avro 1.4.1，这有效：
private static GenericData.Record parseJson(String json, String schema)
throws IOException {
Schema parsedSchema = Schema.parse(schema);
Decoder decoder = new JsonDecoder(parsedSchema, json);
DatumReader<GenericData.Record> reader =
new GenericDatumReader<>(parsedSchema);
return reader.read(null, decoder);
}
Might need some tweaks for later Avro versions.
可能需要对以后的 Avro 版本进行一些调整。
回答by Raman
For anyone who uses Avro - 1.8.2, JsonDecoderis not directly instantiable outside the package org.apache.avro.ionow. You can use DecoderFactoryfor it as shown in the following code:
对于使用 Avro - 1.8.2 的任何人，JsonDecoder现在不能直接在包外实例化org.apache.avro.io。您可以使用DecoderFactory它，如以下代码所示：
String schemaStr = "<some json schema>";
String genericRecordStr = "<some json record>";
Schema.Parser schemaParser = new Schema.Parser();
Schema schema = schemaParser.parse(schemaStr);
DecoderFactory decoderFactory = new DecoderFactory();
Decoder decoder = decoderFactory.jsonDecoder(schema, genericRecordStr);
DatumReader<GenericData.Record> reader =
new GenericDatumReader<>(schema);
GenericRecord genericRecord = reader.read(null, decoder);
