scala Apache Spark 错误：未找到：值 sqlContext
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/42993521/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Apache spark error: not found: value sqlContext
提问by SoakingHummer
I am trying to set up spark in Windows 10. Initially, I faced this errorwhile starting and the solution in the link helped. Now I am still not able to run import sqlContext.sqlas it still throws me an error
我正在尝试在 Windows 10 中设置 spark。最初，我在启动时遇到了这个错误，链接中的解决方案有所帮助。现在我仍然无法运行，import sqlContext.sql因为它仍然向我抛出错误
----------------------------------------------------------------
Fri Mar 24 12:07:05 IST 2017:
Booting Derby version The Apache Software Foundation - Apache Derby - 10.12.1.1 - (1704137): instance a816c00e-015a-ff08-6530-00000ac1cba8
on database directory C:\metastore_db with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon@37606fee
Loaded from file:/F:/Soft/spark/spark-2.1.0-bin-hadoop2.7/bin/../jars/derby-10.12.1.1.jar
java.vendor=Oracle Corporation
java.runtime.version=1.8.0_101-b13
user.dir=C:\
os.name=Windows 10
os.arch=amd64
os.version=10.0
derby.system.home=null
Database Class Loader started - derby.database.classpath=''
17/03/24 12:07:09 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Spark context Web UI available at http://10.128.18.22:4040
Spark context available as 'sc' (master = local[*], app id = local-1490337421381).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.0
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101)
Type in expressions to have them evaluated.
Type :help for more information.
scala> import sqlContext.sql
<console>:23: error: not found: value sqlContext
import sqlContext.sql
^
回答by Balaji Reddy
Spark context available as 'sc' (master = local[*], app id = local-1490337421381).
Spark session available as 'spark'.
Spark 上下文可用作“sc”（master = local[*]，app id = local-1490337421381）。
Spark 会话可用作“火花”。
In Spark 2.0.x, the entry point of Spark is SparkSessionand that is available in Spark shell as spark, so try this way:
在 Spark 2.0.x 中，Spark 的入口点是SparkSession，它在 Spark shell 中可用spark，因此请尝试以下方式：
spark.sqlContext.sql(...)
You can also create your Spark Context like this
你也可以像这样创建你的 Spark Context
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
First option is my choice as Spark shell has already created one for you, so make use of it.
第一个选项是我的选择，因为 Spark shell 已经为您创建了一个，所以请使用它。
回答by aaa90210
If you are on Cloudera and have this issue, the solution from this Github ticket worked for me (https://github.com/cloudera/clusterdock/issues/30):
如果您在 Cloudera 上遇到此问题，则此 Github 票证中的解决方案对我有用（https://github.com/cloudera/clusterdock/issues/30）：
The root user (who you're running as when you start spark-shell) has no user directory in HDFS. If you create one (sudo -u hdfs hdfs dfs -mkdir /user/root followed by sudo -u hdfs dfs -chown root:root /user/root), this should be fixed.
root 用户（您在启动 spark-shell 时运行的用户）在 HDFS 中没有用户目录。如果您创建一个（sudo -u hdfs hdfs dfs -mkdir /user/root 后跟 sudo -u hdfs dfs -chown root:root /user/root），这应该是固定的。
I.e. create a user home directory for the user running spark-shell. This fixed it for me.
即为运行 spark-shell 的用户创建一个用户主目录。这为我修好了。
回答by Gsquare
Since you are using Spark 2.1 you'll have to use the SparkSessionobject. You can get a reference to SparkContextfrom the SparkSessionobject
由于您使用的是 Spark 2.1，因此您必须使用该SparkSession对象。您可以SparkContext从SparkSession对象中获取引用
var sSession = org.apache.spark.sql.SparkSession.getOrCreate();
var sContext = sSession.sparkContext;
