scala 从数据框火花中删除一列

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/41763227/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-22 09:01:46  来源:igfitidea点击:

remove a column from a dataframe spark

scaladataframeapache-sparkapache-spark-sql

提问by Count

I have a Spark dataframe with a very large number of columns. I want to remove two columns from it to get a new dataframe.

我有一个包含大量列的 Spark 数据框。我想从中删除两列以获取新的数据框。

Had there been fewer columns, I could have used the select method in the API like this:

如果列较少,我可以像这样在 API 中使用 select 方法:

pcomments = pcomments.select(pcomments.col("post_id"),pcomments.col("comment_id"),pcomments.col("comment_message"),pcomments.col("user_name"),pcomments.col("comment_createdtime"));

But since picking columns from a long list is a tedious task, is there a workaround?

但是由于从长列表中挑选列是一项乏味的任务,是否有解决方法?

回答by SanthoshPrasad

Use dropmethod and withColumnRenamedmethods.

使用drop方法和withColumnRenamed方法。

Example:

例子:

    val initialDf= ....

    val dfAfterDrop=initialDf.drop("column1").drop("coumn2")

    val dfAfterColRename= dfAfterDrop.withColumnRenamed("oldColumnName","new ColumnName")

回答by Manoj Kumar Dhakad

Try this:

试试这个:

val initialDf = ...

val dfAfterDropCols = initialDf.drop("column1", "coumn2")