scala build.sbt：如何添加火花依赖
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/37958158/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
build.sbt: how to add spark dependencies
提问by Bobby
Hello I am trying to download spark-core, spark-streaming, twitter4j, and spark-streaming-twitterin the build.sbt file below:
你好我想下载spark-core，spark-streaming，twitter4j，和spark-streaming-twitter下面的build.sbt文件：
name := "hello"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.6.1"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.4.1"
libraryDependencies ++= Seq(
"org.twitter4j" % "twitter4j-core" % "3.0.3",
"org.twitter4j" % "twitter4j-stream" % "3.0.3"
)
libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10" % "0.9.0-incubating"
I simply took this libraryDependenciesonline so I am not sure which versions, etc. to use.
我只是把libraryDependencies它放到网上，所以我不确定要使用哪个版本等。
Can someone please explain to me how I should fix this .sbt files. I spent a couple hours trying to figure it out but none of the suggesstion worked. I installed scalathrough homebrew and I am on version 2.11.8
有人可以向我解释我应该如何修复这个 .sbt 文件。我花了几个小时试图弄清楚，但没有一个建议奏效。我scala通过自制软件安装，我在版本2.11.8
All of my errors were about:
我所有的错误都是关于：
Modules were resolved with conflicting cross-version suffixes.
回答by marcospereira
The problem is that you are mixing Scala 2.11 and 2.10 artifacts. You have:
问题是您正在混合 Scala 2.11 和 2.10 工件。你有：
scalaVersion := "2.11.8"
And then:
接着：
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.4.1"
Where the 2.10artifact is being required. You are also mixing Spark versions instead of using a consistent version:
需要2.10工件的地方。您还混合了 Spark 版本，而不是使用一致的版本：
// spark 1.6.1
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.6.1"
// spark 1.4.1
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.4.1"
// spark 0.9.0-incubating
libraryDependencies += "org.apache.spark" % "spark-streaming-twitter_2.10" % "0.9.0-incubating"
Here is a build.sbtthat fixes both problems:
这是build.sbt解决两个问题的方法：
name := "hello"
version := "1.0"
scalaVersion := "2.11.8"
val sparkVersion = "1.6.1"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion,
"org.apache.spark" %% "spark-streaming" % sparkVersion,
"org.apache.spark" %% "spark-streaming-twitter" % sparkVersion
)
You also don't need to manually add twitter4jdependencies since they are added transitively by spark-streaming-twitter.
您也不需要手动添加twitter4j依赖项，因为它们是由spark-streaming-twitter.
回答by AlexPes
It works for me:
这个对我有用：
name := "spark_local"
version := "0.1"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"org.twitter4j" % "twitter4j-core" % "3.0.5",
"org.twitter4j" % "twitter4j-stream" % "3.0.5",
"org.apache.spark" %% "spark-core" % "2.0.0",
"org.apache.spark" %% "spark-sql" % "2.0.0",
"org.apache.spark" %% "spark-mllib" % "2.0.0",
"org.apache.spark" %% "spark-streaming" % "2.0.0"
)
