git 尝试运行 Spark 时出错
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/30139951/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Error while trying to run Spark
提问by SH_V95
I am new to spark. I have downloaded the spark version 1.3.1 prebuilt for hadoop version 2.6 . I extracted and navigated to the folder and typed in the below command : ./bin/spark-shell for which i get an error which says spark-shell command not found. I did the same on windows using git bash for which I get an error saying
我是新来的火花。我已经下载了为 hadoop 版本 2.6 预构建的 spark 版本 1.3.1 。我提取并导航到该文件夹并输入以下命令:./bin/spark-shell 我收到一个错误,提示未找到 spark-shell 命令。我在 Windows 上使用 git bash 做了同样的事情,我得到一个错误说
spark-submit : line 26 tput command not found
spark-submit:找不到第 26 行 tput 命令
is there something else I need to do before trying to run spark?
在尝试运行 spark 之前,我还需要做些什么吗?
回答by Marko Bonaci
On Windows, in regular cmd prompt, use spark-shell.cmd
.
On Linux, in terminal, cd
to your Spark root (yours should be named spark-1.3.1-bin-hadoop2.6
if you kept the original name) and then execute:
在 Windows 上,在常规 cmd 提示符下,使用spark-shell.cmd
.
在 Linux 上,在终端中,cd
到您的 Spark 根目录(spark-1.3.1-bin-hadoop2.6
如果您保留原始名称,则应命名您的根目录),然后执行:
./bin/spark-shell
Have you recently changed your .bash_profile? Any problems with other commands? Try just typing e.g. tar
in your shell. All good or not?
您最近更改了 .bash_profile 吗?其他命令有问题吗?试着tar
在你的shell 中输入例如。都好不好?
EDIT(after the first comment below):
编辑(在下面的第一条评论之后):
Here's how to start the REPL on Linux(logging level set to errors only).Spark
is just a symlink to a Spark version I want to use, ignore that and take it as your Spark home dir:
这是在Linux上启动 REPL 的方法(日志级别仅设置为错误)。Spark
只是我想使用的 Spark 版本的符号链接,忽略它并将其作为您的 Spark 主目录:
And here's Windows:
这是Windows:
You almost cannot do anything wrong how straightforward it is :)
你几乎不会做错任何事,它是多么简单:)
回答by Jmv Jmv
In linux
在 linux 中
- Install scala
- Install git
- Build Spark
- 安装 Scala
- 安装 git
- 构建火花
Build Spark:
构建火花:
$ wget http://archive.apache.org/dist/spark/spark-1.5.0/spark-1.5.0-bin-hadoop2.6.tgz
$ tar xvf spark-1.5.0-bin-hadoop2.6.tgz
$ sudo mv spark-1.5.0-bin-hadoop2.6 /usr/local/spark
$ /usr/local/spark/bin/spark-shell
回答by make
After extracting and navigating to the folder you need to build the Apache Spark by this command:
解压并导航到您需要通过以下命令构建 Apache Spark 的文件夹后:
mvn -DskipTests clean package
where you need to install maven(mvn) by
您需要通过以下方式安装 maven(mvn)
sudo apt-get install maven