scala 如何在 IntelliJ IDEA 中使用 SBT 构建 Uber JAR（Fat JAR）？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/28459333/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to build an Uber JAR (Fat JAR) using SBT within IntelliJ IDEA?
提问by Yves M.
I'm using SBT (within IntelliJ IDEA) to build a simple Scala project.
我正在使用 SBT（在 IntelliJ IDEA 中）构建一个简单的 Scala 项目。
I would like to know what is the simplest wayto build an Uber JARfile (aka Fat JAR, Super JAR).
我想知道构建Uber JAR文件（又名 Fat JAR、Super JAR）的最简单方法是什么。
I'm currently using SBT but when I'm submiting my JAR file to Apache SparkI get the following error:
我目前正在使用 SBT，但是当我将 JAR 文件提交到Apache Spark 时，出现以下错误：
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
线程“main”中的异常 java.lang.SecurityException：Manifest 主要属性的签名文件摘要无效
Or this error during compilation time:
或者在编译期间出现这个错误：
java.lang.RuntimeException: deduplicate: different file contents found in the following:
PATH\DEPENDENCY.jar:META-INF/DEPENDENCIES
PATH\DEPENDENCY.jar:META-INF/MANIFEST.MF
java.lang.RuntimeException：重复数据删除：在以下位置找到不同的文件内容：
PATH\DEPENDENCY.jar:META-INF/DEPENDENCIES
PATH\DEPENDENCY.jar:META-INF/MANIFEST.MF
It looks likeit is because some of my dependencies include signature files (META-INF) which needs to be removed in the final Uber JAR file.
它看起来像那是因为我的一些依赖包括需要在尤伯杯决赛JAR文件被删除签名文件（META-INF）。
I tried to use the sbt-assemblyplugin like that:
我尝试像这样使用sbt-assembly插件：
/project/assembly.sbt
/项目/程序集.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
/project/plugins.sbt
/project/plugins.sbt
logLevel := Level.Warn
/build.sbt
/build.sbt
lazy val commonSettings = Seq(
name := "Spark-Test"
version := "1.0"
scalaVersion := "2.11.4"
)
lazy val app = (project in file("app")).
settings(commonSettings: _*).
settings(
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.2.0",
"org.apache.spark" %% "spark-streaming" % "1.2.0",
"org.apache.spark" % "spark-streaming-twitter_2.10" % "1.2.0"
)
)
When I click "Build Artifact..." in IntelliJ IDEA I get a JAR file. But I end up with the same error...
当我在 IntelliJ IDEA 中单击“ Build Artifact...”时，我得到一个 JAR 文件。但我最终遇到了同样的错误......
I'm new to SBT and not very experimented with IntelliJ IDE.
我是 SBT 的新手，对 IntelliJ IDE 的试验并不多。
Thanks.
谢谢。
回答by Yves M.
Finally I totally skip using IntelliJ IDEA to avoid generating noise in my global understanding :)
最后，我完全跳过使用 IntelliJ IDEA 以避免在我的全局理解中产生噪音:)
I started reading the official SBT tutorial.
我开始阅读官方的 SBT 教程。
I created my project with the following file structure :
我使用以下文件结构创建了我的项目：
my-project/project/assembly.sbt
my-project/src/main/scala/myPackage/MyMainObject.scala
my-project/build.sbt
Added the sbt-assemblypluginin my assembly.sbtfile. Allowing me to build a fat JAR :
在我的assembly.sbt文件中添加了sbt-assembly插件。允许我构建一个胖 JAR：
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
My minimal build.sbtlooks like :
我的最小build.sbt看起来像：
lazy val root = (project in file(".")).
settings(
name := "my-project",
version := "1.0",
scalaVersion := "2.11.4",
mainClass in Compile := Some("myPackage.MyMainObject")
)
val sparkVersion = "1.2.0"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",
"org.apache.spark" %% "spark-streaming-twitter" % sparkVersion
)
// META-INF discarding
mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
{
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
}
Note: The % "provided"means not to include the dependency in the final fat JAR (those libraries are already included in my workers)
注意：% "provided"在最终的胖 JAR 中不包含依赖项的方法（这些库已经包含在我的工作人员中）
Note: META-INF discarding inspired by this answser.
注意： META-INF 丢弃受此 answser 启发。
Now I can build my fat JAR using SBT (how to install it) by running the following command in my /my-projectroot folder:
现在我可以通过在我的/my-project根文件夹中运行以下命令来使用 SBT（如何安装它）构建我的胖 JAR ：
sbt assembly
My fat JAR is now located in the new generated /targetfolder :
我的胖 JAR 现在位于新生成的/target文件夹中：
/my-project/target/scala-2.11/my-project-assembly-1.0.jar
Hope that helps someone else.
希望能帮助别人。
For those who wants to embeed SBT within IntelliJ IDE: How to run sbt-assembly tasks from within IntelliJ IDEA?
对于那些想要在 IntelliJ IDE 中嵌入 SBT 的人：如何从 IntelliJ IDEA 中运行 sbt-assembly 任务？
回答by Ajay Gupta
3 Step Process For Building Uber JAR/Fat JAR in IntelliJ Idea:
在 IntelliJ Idea 中构建 Uber JAR/Fat JAR 的 3 步过程：
Uber JAR/Fat JAR: JAR file having all external libraray dependencies in it.
Uber JAR/Fat JAR：包含所有外部库依赖项的 JAR 文件。
Adding SBT Assembly plugin in IntelliJ Idea
Go to ProjectName/project/target/plugins.sbtfile and add this line
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")Adding Merge,Discard and Do Not Add strategy in build.sbt
Go to ProjectName/build.sbtfile and add the Strategy for Packaging of an Uber JAR
Merge Strategy :If there is conflict in two packages about a version of library then which one to pack in Uber JAR.
Discard Strategy :To remove some files from library which you do not want to package in Uber JAR.
Do not Add Strategy :Do not add some package to Uber JAR.
For ex:spark-corewill be already present at your Spark Cluster.So we should not package this in Uber JARMerge Strategy and Discard Strategy Basic Code :
assemblyMergeStrategy in assembly := { case PathList("META-INF", xs @ _*) => MergeStrategy.discard case x => MergeStrategy.first }So you are asking to discard META-INF files using this command
MergeStrategy.discardand for rest of the files you are taking the first occurrenceof library file if there is any conflict by using this commandMergeStrategy.first.Do not Add Strategy Basic Code :
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.1" %"provided"If we do not want to add the spark-core to our Uber JAR file as it will be already on our clutser, so we are adding the
% "provided"at end of it library dependency.Building Uber JAR with all its dependencies
In terminal type
sbt assemblyfor building up the package
在 IntelliJ Idea 中添加 SBT Assembly 插件
转到ProjectName/project/target/plugins.sbt文件并添加这一行
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")在 build.sbt 中添加合并、丢弃和不添加策略
转到ProjectName/build.sbt文件并添加 Uber JAR 打包策略
合并策略：如果关于一个库版本的两个包存在冲突，那么在 Uber JAR 中打包哪个。
丢弃策略：从库中删除一些您不想打包到 Uber JAR 中的文件。
不要添加策略：不要向 Uber JAR 添加一些包。
例如：spark-core将已经存在于您的 Spark 集群中。所以我们不应将其打包在 Uber JAR 中合并策略和丢弃策略基本代码：
assemblyMergeStrategy in assembly := { case PathList("META-INF", xs @ _*) => MergeStrategy.discard case x => MergeStrategy.first }因此，您要求使用此命令丢弃 META-INF 文件
MergeStrategy.discard，对于其余文件，如果使用此命令有任何冲突，则您将首次出现库文件MergeStrategy.first。不要添加策略基本代码：
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.1" %"provided"如果我们不想将 spark-core 添加到我们的 Uber JAR 文件中，因为它已经在我们的 clutser 上，所以我们
% "provided"在它的末尾添加库依赖项。构建 Uber JAR 及其所有依赖项
在
sbt assembly用于构建包的终端类型中
Voila!!! Uber JAR is built. JAR will be in ProjectName/target/scala-XX
瞧！！！Uber JAR 构建完成。JAR 将在ProjectName/target/scala-XX
回答by ARMV
Add the following line to your project/plugins.sbt
将以下行添加到您的项目/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
Add the following to your build.sbt
将以下内容添加到您的 build.sbt
mainClass in assembly := some("package.MainClass")
assemblyJarName := "desired_jar_name_after_assembly.jar"
val meta = """META.INF(.)*""".r
assemblyMergeStrategy in assembly := {
case PathList("javax", "servlet", xs @ _*) => MergeStrategy.first
case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
case n if n.startsWith("reference.conf") => MergeStrategy.concat
case n if n.endsWith(".conf") => MergeStrategy.concat
case meta(_) => MergeStrategy.discard
case x => MergeStrategy.first
}
The Assembly merge strategy is used to resolve conflicts occurred when creating fat jar.
Assembly合并策略用于解决创建fat jar时发生的冲突。
