scala 从 Apache SQL Spark 中删除临时表

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/32376066/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-22 07:33:29  来源:igfitidea点击:

Remove Temporary Tables from Apache SQL Spark

scalaapache-sparkapache-spark-sqlapache-zeppelin

提问by Softwaremaker

I have registertemptablein Apache Sparkusing Zeppelinbelow:

registertemptable在下面Apache Spark使用Zeppelin

val hvacText = sc.textFile("...")

case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String)

val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map(
    s => Hvac(s(0), 
            s(1),
            s(2).toInt,
            s(3).toInt,
            s(6))).toDF()

hvac.registerTempTable("hvac")

After I have done with my queries with this temp table, how do I remove it ?

使用此临时表完成查询后,如何删除它?

I checked all docs and it seems I am getting nowhere.

我检查了所有文档,似乎一无所获。

Any guidance ?

任何指导?

回答by zero323

Spark 2.x

火花2.x

For temporary views you can use Catalog.dropTempView:

对于临时视图,您可以使用Catalog.dropTempView

spark.catalog.dropTempView("df")

For global views you can use Catalog.dropGlobalTempView:

对于全局视图,您可以使用Catalog.dropGlobalTempView

spark.catalog.dropGlobalTempView("df")

Both methods are safe to call if view doesn't exist and, since Spark 2.1, return boolean indicating if the operation succeed.

如果视图不存在,两种方法都可以安全调用,并且从 Spark 2.1 开始,返回布尔值,指示操作是否成功。

Spark 1.x

火花1.x

You can use SQLContext.dropTempTable:

您可以使用SQLContext.dropTempTable

scala.util.Try(sqlContext.dropTempTable("df"))

It can be still used in Spark 2.0, but delegates processing to Catalog.dropTempViewand is safe to use if table doesn't exist.

它仍然可以在 Spark 2.0 中使用,但将处理委托给Catalog.dropTempView并且如果表不存在则可以安全使用。

回答by desaiankitb

in new ver (2.0 and latest) of spark. one should use: createOrReplaceTempViewin place of registerTempTable(depricated) and corresponding method to deallocate is: dropTempView

在 spark 的新版本(2.0 和最新版本)中。一个应该使用:createOrReplaceTempView代替registerTempTable(depriated)和相应的解除分配方法是:dropTempView

spark.catalog.dropTempView("temp_view_name") //drops the table

回答by ??? Hyung Sung Shim

If you want to remove your temp table on zeppelin, try like this.

如果您想删除 zeppelin 上的临时表,请尝试这样。

sqlc.dropTempTable("hvac")

or

或者

%sql DROP VIEW hvac

And you can get the informations you need from spark API Docs(http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package)

你可以从 spark API Docs( http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package)获取你需要的信息

回答by msrv499

You can use sql drop table/view statement to remove it like below

您可以使用 sql drop table/view 语句将其删除,如下所示

spark.sql("drop view hvac");