scala 从 Apache SQL Spark 中删除临时表
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/32376066/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Remove Temporary Tables from Apache SQL Spark
提问by Softwaremaker
I have registertemptablein Apache Sparkusing Zeppelinbelow:
我registertemptable在下面Apache Spark使用Zeppelin:
val hvacText = sc.textFile("...")
case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String)
val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map(
s => Hvac(s(0),
s(1),
s(2).toInt,
s(3).toInt,
s(6))).toDF()
hvac.registerTempTable("hvac")
After I have done with my queries with this temp table, how do I remove it ?
使用此临时表完成查询后,如何删除它?
I checked all docs and it seems I am getting nowhere.
我检查了所有文档,似乎一无所获。
Any guidance ?
任何指导?
回答by zero323
Spark 2.x
火花2.x
For temporary views you can use Catalog.dropTempView:
对于临时视图,您可以使用Catalog.dropTempView:
spark.catalog.dropTempView("df")
For global views you can use Catalog.dropGlobalTempView:
对于全局视图,您可以使用Catalog.dropGlobalTempView:
spark.catalog.dropGlobalTempView("df")
Both methods are safe to call if view doesn't exist and, since Spark 2.1, return boolean indicating if the operation succeed.
如果视图不存在,两种方法都可以安全调用,并且从 Spark 2.1 开始,返回布尔值,指示操作是否成功。
Spark 1.x
火花1.x
You can use SQLContext.dropTempTable:
您可以使用SQLContext.dropTempTable:
scala.util.Try(sqlContext.dropTempTable("df"))
It can be still used in Spark 2.0, but delegates processing to Catalog.dropTempViewand is safe to use if table doesn't exist.
它仍然可以在 Spark 2.0 中使用,但将处理委托给Catalog.dropTempView并且如果表不存在则可以安全使用。
回答by desaiankitb
in new ver (2.0 and latest) of spark.
one should use: createOrReplaceTempViewin place of registerTempTable(depricated)
and corresponding method to deallocate is: dropTempView
在 spark 的新版本(2.0 和最新版本)中。一个应该使用:createOrReplaceTempView代替registerTempTable(depriated)和相应的解除分配方法是:dropTempView
spark.catalog.dropTempView("temp_view_name") //drops the table
回答by ??? Hyung Sung Shim
If you want to remove your temp table on zeppelin, try like this.
如果您想删除 zeppelin 上的临时表,请尝试这样。
sqlc.dropTempTable("hvac")
or
或者
%sql DROP VIEW hvac
And you can get the informations you need from spark API Docs(http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package)
你可以从 spark API Docs( http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package)获取你需要的信息
回答by msrv499
You can use sql drop table/view statement to remove it like below
您可以使用 sql drop table/view 语句将其删除,如下所示
spark.sql("drop view hvac");

