scala 错误实用程序：线程 SparkListenerBus 中未捕获的异常
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/28362341/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
ERROR Utils: Uncaught exception in thread SparkListenerBus
提问by Bobs
I try to execute simple project with Apache Spark. This is my code SimpleApp.scala
我尝试使用 Apache Spark 执行简单的项目。这是我的代码 SimpleApp.scala
/* SimpleApp.scala */
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
object SimpleApp {
def main(args: Array[String]) {
val logFile = "/home/hduser/spark-1.2.0-bin-hadoop2.4/README.md" // Should be some file on your system
// val conf = new SparkConf().setAppName("Simple Application")
val sc = new SparkContext("local", "Simple Job", "/home/hduser/spark-1.2.0-bin-hadoop2.4/")
val logData = sc.textFile(logFile, 2).cache()
val numAs = logData.filter(line => line.contains("hadoop")).count()
val numBs = logData.filter(line => line.contains("see")).count()
println("Lines with hadoop: %s, Lines with see: %s".format(numAs, numBs))
}
}
when I manually send this job to Spark with command line : /home/hduser/spark-1.2.0-hadoop-2.4.0/bin/spark-submit --class "SimpleApp" --master local[4] target/scala-2.10/simple-project_2.10-1.0.jarit's run successfully.
当我使用命令行手动将此作业发送到 Spark 时：/home/hduser/spark-1.2.0-hadoop-2.4.0/bin/spark-submit --class "SimpleApp" --master local[4] target/scala-2.10/simple-project_2.10-1.0.jar它已成功运行。
if I run with sbt runand with the service apache spark is running, it's success, but in the end of log it give error like this :
如果我运行sbt run并使用服务 apache spark 运行，它是成功的，但在日志结束时它会给出如下错误：
15/02/06 15:56:49 ERROR Utils: Uncaught exception in thread SparkListenerBus
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:317)
at org.apache.spark.scheduler.LiveListenerBus$$anon$$anonfun$run.apply$mcV$sp(LiveListenerBus.scala:48)
at org.apache.spark.scheduler.LiveListenerBus$$anon$$anonfun$run.apply(LiveListenerBus.scala:47)
at org.apache.spark.scheduler.LiveListenerBus$$anon$$anonfun$run.apply(LiveListenerBus.scala:47)
at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1460)
at org.apache.spark.scheduler.LiveListenerBus$$anon.run(LiveListenerBus.scala:46)
15/02/06 15:56:49 ERROR ContextCleaner: Error in cleaning thread
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning.apply$mcV$sp(ContextCleaner.scala:136)
at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning.apply(ContextCleaner.scala:134)
at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning.apply(ContextCleaner.scala:134)
at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1460)
at org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:133)
at org.apache.spark.ContextCleaner$$anon.run(ContextCleaner.scala:65)
Any wrong in my code? Thanks in advance. I use apache spark 1.2.0-bin-hadoop-2.4, scala 2.10.4
我的代码有什么问题吗？提前致谢。我使用 apache spark 1.2.0-bin-hadoop-2.4，scala 2.10.4
采纳答案by Shyamendra Solanki
According this mail archive, i.e.:
根据此邮件存档，即：
Hi Haoming,
You can safely disregard this error. This is printed at the end of the execution when we clean up and kill the daemon context cleaning thread. In the future it would be good to silence this particular message, as it may be confusing to users.
Andrew
嗨，浩明，
您可以放心地忽略此错误。当我们清理并杀死守护进程上下文清理线程时，它会在执行结束时打印。将来最好将此特定消息静音，因为它可能会使用户感到困惑。
安德鲁
the error could be disregarded.
可以忽略错误。
回答by M.Rez
The SparkContextor SparkSession(Spark >= 2.0.0) should be stopped when the Spark code is run by adding sc.stopor spark.stop(Spark >= 2.0.0) at the end of the code.
的SparkContext或SparkSession当火花码是通过将运行（火花> = 2.0.0）应该被停止sc.stop或spark.stop（火花> = 2.0.0）的代码的末尾。
