scala 无法解析 Spark Dataframe 中的列(数字列名称)

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/42698322/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-22 09:07:25  来源:igfitidea点击:

Cannot resolve column (numeric column name) in Spark Dataframe

scalaapache-sparkspark-dataframe

提问by Marsellus Wallace

This is my data:

这是我的数据:

scala> data.printSchema
root
 |-- 1.0: string (nullable = true)
 |-- 2.0: string (nullable = true)
 |-- 3.0: string (nullable = true)

This doesn't work :(

这不起作用:(

scala> data.select("2.0").show

Exception:

例外:

org.apache.spark.sql.AnalysisException: cannot resolve '`2.0`' given input columns: [1.0, 2.0, 3.0];;
'Project ['2.0]
+- Project [_1#5608 AS 1.0#5615, _2#5609 AS 2.0#5616, _3#5610 AS 3.0#5617]
   +- LocalRelation [_1#5608, _2#5609, _3#5610]
        ...

Try this at home (I'm running on the shell v_2.1.0.5)!

在家里试试这个(我在 shell v_2.1.0.5 上运行)!

val data = spark.createDataFrame(Seq(
  ("Hello", ", ", "World!")
)).toDF("1.0", "2.0", "3.0")
data.select("2.0").show

回答by Psidom

You can use backticksto escape the dot, which is reserved for accessing columns for struct type:

您可以使用反引号来转义点,该点保留用于访问结构类型的列:

data.select("`2.0`").show
+---+
|2.0|
+---+
| , |
+---+

回答by Tawkir

The problem is you can not add dot character in the column name while selecting from dataframe. You can have a look at this question, kind of similar.

问题是您无法在从数据框中选择时在列名中添加点字符。你可以看看这个问题,有点类似。

val data = spark.createDataFrame(Seq(
  ("Hello", ", ", "World!")
)).toDF("1.0", "2.0", "3.0")
data.select(sanitize("2.0")).show

def sanitize(input: String): String = s"`$input`"