scala SBT 程序集 - 重复数据删除错误和排除错误
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/27198216/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
SBT Assembly - Deduplicate error & Exclude error
提问by macha
hey guys I am trying to build a JAR with dependencies using sbt-assembly. But I am running into this error again and again. I have tried multiple different things but I end up here. I am pretty new to SBT and wanted to get some help on this one. Here are the build.sbt & assembly.sbt files.
嘿伙计们，我正在尝试使用 sbt-assembly 构建具有依赖项的 JAR。但我一次又一次地遇到这个错误。我尝试了多种不同的东西，但最终还是到了这里。我对 SBT 很陌生，想在这方面获得一些帮助。这是 build.sbt 和 assembly.sbt 文件。
build.sbt
生成.sbt
seq(assemblySettings: _*)
name := "StreamTest"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"
project/assembly.sbt
项目/程序集.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
When I run the sbt assembly command, I am getting this error below.
当我运行 sbt assembly 命令时，出现以下错误。
[info] Including: joda-time-2.5.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'about_files/LICENSE.txt' with strategy 'rename'
[warn] Merging 'about_files/NOTICE.txt' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE.txt' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE' with strategy 'rename'
[warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename'
[warn] Merging 'META-INF/maven/org.xerial.snappy/snappy-java/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/license' with strategy 'rename'
[warn] Merging 'about.html' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/README.txt' with strategy 'rename'
[warn] Merging 'LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard'
java.lang.RuntimeException: deduplicate: different file contents found in the following:
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$$applyStrategy(Plugin.scala:253)
at sbtassembly.Plugin$Assembly$$anonfun.apply(Plugin.scala:270)
at sbtassembly.Plugin$Assembly$$anonfun.apply(Plugin.scala:267)
at scala.collection.TraversableLike$$anonfun$flatMap.apply(TraversableLike.scala:251)
at scala.collection.TraversableLike$$anonfun$flatMap.apply(TraversableLike.scala:251)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272)
at sbtassembly.Plugin$Assembly$.x$lzycompute(Plugin.scala:172)
at sbtassembly.Plugin$Assembly$.x(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.stratMapping(Plugin.scala:170)
at sbtassembly.Plugin$Assembly$.inputs$lzycompute(Plugin.scala:214)
at sbtassembly.Plugin$Assembly$.inputs(Plugin.scala:204)
at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230)
at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask.apply(Plugin.scala:373)
at sbtassembly.Plugin$Assembly$$anonfun$assemblyTask.apply(Plugin.scala:370)
at scala.Function1$$anonfun$compose.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon.work(System.scala:63)
at sbt.Execute$$anonfun$submit$$anonfun$apply.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$$anonfun$apply.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$$anonfun.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
[error] Total time: 23 s, completed Nov 28, 2014 9:32:53 PM
sbt-version
sbt 版本
0.13.6
EDIT
编辑
Now after looking around I have made another change to exclude any dependencies, as part of another question on stackoverflow.
现在，在环顾四周之后，我做了另一个更改以排除任何依赖项，作为关于 stackoverflow 的另一个问题的一部分。
Updated build.sbt
更新了 build.sbt
seq(assemblySettings: _*)
name := "StreamTest"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"
libraryDependencies ++= Seq(
exclude("org.eclipse.jetty.orbit", "javax.servlet").
exclude("org.eclipse.jetty.orbit", "javax.transaction").
exclude("org.eclipse.jetty.orbit", "javax.mail").
exclude("org.eclipse.jetty.orbit", "javax.activation").
exclude("commons-beanutils", "commons-beanutils-core").
exclude("commons-collections", "commons-collections").
exclude("commons-collections", "commons-collections").
exclude("com.esotericsoftware.minlog", "minlog")
)
When I run the assembly command again, this is the error that I get.
当我再次运行程序集命令时，这是我得到的错误。
build.sbt:14: error: not found: value exclude
exclude("org.eclipse.jetty.orbit", "javax.servlet").
^
EDIT 2:
编辑2：
Updated build.sbt
更新了 build.sbt
import AssemblyKeys._
seq(assemblySettings: _*)
name := "SparkStreamingKinesis"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"
assemblyMergeStrategy in assembly := {
case PathList(ps @ _*) if ps.last endsWith ".RSA" => MergeStrategy.first
case x =>
val oldStrategy = (assemblyMergeStrategy in assembly).value
oldStrategy(x)
}
After making the update, here is the error that I am getting. I tried to do an import of assemblyMergeStrategy, but it doesn't look like a class that I can import.
进行更新后，这是我收到的错误。我尝试导入 assemblyMergeStrategy，但它看起来不像我可以导入的类。
build.sbt:21: error: not found: value assemblyMergeStrategy
assemblyMergeStrategy in assembly := {
^
[error] Type error in expression
回答by dk14
Try this one to exclude only ECLIPSEF.RSAfile, because excluding whole dependencies, which your libraries may need, is a bad idea :)
试试这个只排除ECLIPSEF.RSA文件，因为排除你的库可能需要的整个依赖项是一个坏主意:)
Deduplication error in build.sbt when building a fat jar using sbt assembly
使用 sbt 程序集构建胖 jar 时 build.sbt 中的重复数据删除错误
Solution for latest 0.12.0version of assembly plugin:
最新0.12.0版本的程序集插件的解决方案：
assemblyMergeStrategy in assembly := {
case PathList(ps @ _*) if ps.last endsWith ".RSA" => MergeStrategy.first
case x =>
val oldStrategy = (assemblyMergeStrategy in assembly).value
oldStrategy(x)
}
project/assembly.sbt
项目/程序集.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
See https://github.com/sbt/sbt-assembly#merge-strategy
见https://github.com/sbt/sbt-assembly#merge-strategy
P.S. About exclude: you can't just specify excludein libraryDependencies - you need something like this: "log4j" % "log4j" % "1.2.15" exclude("javax.jms", "jms"). See Library Management
PS关于exclude：你不能只指定exclude在libraryDependencies -你需要的东西是这样的："log4j" % "log4j" % "1.2.15" exclude("javax.jms", "jms")。见图书馆管理
回答by Justin Pihony
My problem was that I had two dependencies that had dependencies to the same assemblies, but different versions. I simply had to match up the assemblies.
我的问题是我有两个依赖项，它们依赖于相同的程序集，但版本不同。我只需要匹配程序集。
