scala 如何在代码的任何位置获取当前 SparkSession?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/44502872/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How can I get the current SparkSession in any place of the codes?
提问by PC9527
I have created a session in the main()function, like this:
我在main()函数中创建了一个会话,如下所示:
val sparkSession = SparkSession.builder.master("local[*]").appName("Simple Application").getOrCreate()
Now if I want to configure the application or access the properties, I can use the local variable sparkSessionin the same function.
现在如果我想配置应用程序或访问属性,我可以sparkSession在同一个函数中使用局部变量。
What if I want to access this sparkSessionelsewhere in the same project, like project/module/.../.../xxx.scala. What should I do?
如果我想sparkSession在同一个项目的其他地方访问它,比如project/module/.../.../xxx.scala. 我该怎么办?
回答by Tzach Zohar
Once a session was created (anywhere), you can safely use:
创建会话后(在任何地方),您可以安全地使用:
SparkSession.builder().getOrCreate()
To get the (same) session anywhere in the code, as long as the session is still alive. Spark maintains a single active session so unless it was stopped or crashed, you'll get the same one.
要在代码中的任何位置获取(相同的)会话,只要会话仍然存在。Spark 维护一个活动会话,因此除非它停止或崩溃,否则您将获得相同的会话。
回答by Andrei Boaghe
Since 2.2.0you can access the active SparkSessionthrough:
从2.2.0 开始,您可以通过以下方式访问活动的 SparkSession:
/**
* Returns the active SparkSession for the current thread, returned by the builder.
*
* @since 2.2.0
*/
def getActiveSession: Option[SparkSession] = Option(activeThreadSession.get)
or default SparkSession:
或默认 SparkSession:
/**
* Returns the default SparkSession that is returned by the builder.
*
* @since 2.2.0
*/
def getDefaultSparkSession: Option[SparkSession] = Option(defaultSession.get)
回答by Ramesh Maharjan
When SparkSessionvariable has been defined as
当SparkSession变量被定义为
val sparkSession = SparkSession.builder.master("local[*]").appName("Simple Application").getOrCreate()
This variable is going to point/referto only one SparkSessionas its a val. And you can always pass to different classes for them to access as well as
此变量将point/refer仅将一个SparkSession作为其 a val。你总是可以传递给不同的类让他们访问以及
val newClassCall = new NewClass(sparkSession)
Now you can use the same sparkSessionin that new class as well.
现在,您也可以sparkSession在该新类中使用相同的内容。

