scala 如何使用spark在数据框中创建模式数组
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/39485374/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to create schema Array in data frame with spark
提问by RJK
I have code to create data frame and this works fine if there is no array in my input data.
我有创建数据框的代码，如果我的输入数据中没有数组，这可以正常工作。
I tried using Json data which doen't have array and it runs successfully. My code is
我尝试使用没有数组的 Json 数据并且它运行成功。我的代码是
val vals = sc.parallelize(
"""{"id":"1","name":"alex"}""" ::
Nil
)
val schema = (new StructType)
.add("id", StringType)
.add("name", StringType)
sqlContext.read.schema(schema).json(vals).select($"*").printSchema()
My question is, if I have input data with array like below then how to create schema?
我的问题是，如果我有如下数组的输入数据，那么如何创建模式？
val vals = sc.parallelize(
"""{"id":"1","name":"alex","score":[{"keyword":"read","point":10}]}""" ::
Nil
)
val schema = (new StructType)
.add("id", StringType)
.add("name", StringType)
Thanks.
谢谢。
回答by RJK
Oke, I could have solution in my code.
好的，我可以在我的代码中找到解决方案。
Create schema in array in data frame spark you can this code.
在数据框中创建数组中的模式 spark 您可以使用此代码。
val vals = sc.parallelize(
"""{"id":"1","name":"alex","score":[{"keyword":"read","point":10}]}""" ::
Nil
)
val schema = StructType(
Array(
StructField("id", StringType),
StructField("name", StringType),
StructField("score", ArrayType(StructType(Array(
StructField("keyword", StringType),
StructField("point", IntegerType)
))))
)
)
and you print schema
然后你打印模式
sqlContext.read.schema(schema).json(vals).select($"*").printSchema()
Thanks is resolved
谢谢已解决
