scala 中 Apache Spark 中不支持的文字类型类
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/44805407/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Unsupported literal type class in Apache Spark in scala
提问by Queen
I have the following data:
我有以下数据：
+---------------+-----------+-------------+-----+------+
| time_stamp_0|sender_ip_1|receiver_ip_2|count|attack|
+---------------+-----------+-------------+-----+------+
|06:10:55.881073| 10.0.0.3| 10.0.0.1| 1 | 0|
|06:10:55.881095| 10.0.0.3| 10.0.0.1| 2 | 0|
|06:10:55.881114| 10.0.0.3| 10.0.0.1| 3 | 0|
|06:10:55.881133| 10.0.0.3| 10.0.0.1| 4 | 0|
|06:10:55.881152| 10.0.0.3| 10.0.0.1| 5 | 0|
|06:10:55.881172| 10.0.0.3| 10.0.0.1| 6 | 0|
|06:10:55.881191| 10.0.0.3| 10.0.0.1| 7 | 0|
|06:10:55.881210| 10.0.0.3| 10.0.0.1| 8 | 0|
I need to compare the total standard deviation on count column with itself (with count column) in my dataframe. Here is my code:
我需要在我的数据框中将计数列的总标准偏差与其自身（带有计数列）进行比较。这是我的代码：
val std_dev=Dataframe_addcount.agg(stddev_pop($"count"))
val final_add_count_attack = Dataframe_addcount.withColumn("attack", when($"count" > std_dev , 0).otherwise(1))
However my problem is that, I got the following error:
但是我的问题是，我收到以下错误：
Unsupported literal type class org.apache.spark.sql.Dataset [stddev_pop(count): double]
Could you help me? Thanks a lot.
你可以帮帮我吗？非常感谢。
采纳答案by T. Gaw?da
It's because in when and otherwise you should use values; not std_dev is a DataFrame.
这是因为您应该在何时或以其他方式使用值；不是 std_dev 是一个数据帧。
You can get result:
你可以得到结果：
val stdDevValue = std_dev.head().getDouble(0)
val final_add_count_attack = Dataframe_addcount.withColumn("attack", when($"count" > lit(std_dev), lit(0)).otherwise(lit(1)))
