Python 将“SPARK_HOME”设置为什么?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/30824818/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-19 09:02:03  来源:igfitidea点击:

What to set `SPARK_HOME` to?

pythonapache-sparkpythonpathpysparkapache-zeppelin

提问by A T

Installed apache-maven-3.3.3, scala 2.11.6, then ran:

安装 apache-maven-3.3.3,scala 2.11.6,然后运行:

$ git clone git://github.com/apache/spark.git -b branch-1.4
$ cd spark
$ build/mvn -DskipTests clean package

Finally:

最后:

$ git clone https://github.com/apache/incubator-zeppelin
$ cd incubator-zeppelin/
$ mvn install -DskipTests

Then ran the server:

然后运行服务器:

$ bin/zeppelin-daemon.sh start

Running a simple notebook beginning with %pyspark, I got an error about py4jnot being found. Just did pip install py4j(ref).

运行一个以 开头的简单笔记本%pyspark,我收到一个关于py4j找不到的错误。刚刚做了pip install py4j参考)。

Now I'm getting this error:

现在我收到此错误:

pyspark is not responding Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark.py", line 22, in <module>
    from pyspark.conf import SparkConf
ImportError: No module named pyspark.conf

I've tried setting my SPARK_HOMEto: /spark/python:/spark/python/lib. No change.

我试过将我的设置SPARK_HOME为:/spark/python:/spark/python/lib。没变。

采纳答案by ChromeHearts

Two environment variables are required:

需要两个环境变量:

SPARK_HOME=/spark
PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-VERSION-src.zip:$PYTHONPATH