java JVM 消耗所有 CPU,大多数线程被阻塞。JVM错误?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/3449918/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-30 01:55:27  来源:igfitidea点击:

JVM consumes all CPU, most threads as BLOCKED. JVM bug?

javamultithreadingconcurrencylocking

提问by wishihadabettername

Last night, a server (JBoss 5.1GA, Java(TM) SE Runtime Environment (build 1.6.0_20-b02), Java HotSpot(TM) 64-Bit Server VM, running in a Linux VM on VMWare) suddenly started to use 100% CPU. The app is a fairly typical J2EE business app running Seam, nothing particular about it. The load was extremely low at that time.

昨晚,一个服务器(JBoss 5.1GA、Java(TM) SE Runtime Environment (build 1.6.0_20-b02)、Java HotSpot(TM) 64-Bit Server VM,在VMWare上的Linux VM中运行)突然开始使用100 % 中央处理器。该应用程序是一个相当典型的运行 Seam 的 J2EE 业务应用程序,没有什么特别之处。当时的负荷极低。

I managed to get a thread dump before the process was killed and restarted (the app had to be available). JStack had an exception in its deadlock detection; apart from that most threads were in BLOCKED.

我设法在进程被终止并重新启动之前获得了一个线程转储(应用程序必须可用)。JStack 的死锁检测有异常;除此之外,大多数线程都处于阻塞状态。

Is this a JVM bug? I haven't seen it before. After restart everything was ok, like in the several months since the application has been used on that server (with periodic updates).

这是一个JVM错误吗?我以前没见过。重新启动后一切正常,就像在该服务器上使用该应用程序(定期更新)以来的几个月一样。

Thanks for any suggestions.

感谢您的任何建议。

A slightly edited thread dump (to fit within SO's post size limit) is below. I removed a few identical threads (from the various thread pools inside JBoss/Seam) and left only the top lines from the stack traces.

稍微编辑的线程转储(以适应 SO 的帖子大​​小限制)如下。我删除了一些相同的线程(从 JBoss/Seam 内的各种线程池中),只留下堆栈跟踪中的最上面几行。

EDIT: full thread dumpat http://pastie.org/1083984

编辑:完整的线程转储http://pastie.org/1083984

[~]$ jstack -F 13553
Attaching to process ID 13553, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.3-b02
Deadlock Detection:

java.lang.NullPointerException
        at sun.jvm.hotspot.oops.InstanceKlass.computeSubtypeOf(InstanceKlass.java:426)
        at sun.jvm.hotspot.oops.Klass.isSubtypeOf(Klass.java:137)
        at sun.jvm.hotspot.oops.Oop.isA(Oop.java:92)
        at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:93)
        at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:39)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:52)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
        at sun.tools.jstack.JStack.main(JStack.java:84)
Can't print deadlocks:null
Thread 26208: (state = BLOCKED)


Thread 25250: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 25249: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 21240: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21140: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21139: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21138: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 19380: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19377: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19361: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)

Thread 19343: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19317: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 18995: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)

Thread 18986: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)

Thread 17659: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17658: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17653: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17631: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14977: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14796: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14609: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14437: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14248: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14232: (state = BLOCKED)
 - java.util.Arrays.copyOf(char[], int) @bci=1, line=2882 (Compiled frame)
 - java.lang.AbstractStringBuilder.expandCapacity(int) @bci=33, line=100 (Compiled frame)

Thread 14040: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
 - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=384 (Interpreted frame)

Thread 13858: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)

Thread 13815: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

 Thread 13814: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13813: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)


Thread 13812: (state = BLOCKED)
Error occurred during stack walking:
sun.jvm.hotspot.utilities.AssertionFailure: range check
        at sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
        at sun.jvm.hotspot.runtime.RegisterMap.<init>(RegisterMap.java:109)
        at sun.jvm.hotspot.runtime.x86.X86RegisterMap.<init>(X86RegisterMap.java:39)
        at sun.jvm.hotspot.runtime.x86.X86RegisterMap.clone(X86RegisterMap.java:43)
        at sun.jvm.hotspot.runtime.VFrame.<init>(VFrame.java:37)
        at sun.jvm.hotspot.runtime.JavaVFrame.<init>(JavaVFrame.java:45)
        at sun.jvm.hotspot.runtime.CompiledVFrame.<init>(CompiledVFrame.java:43)
        at sun.jvm.hotspot.runtime.VFrame.newVFrame(VFrame.java:77)
        at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:129)
        at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:146)
        at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:231)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
        at sun.tools.jstack.JStack.main(JStack.java:84)


Thread 13811: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13810: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13809: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.jms.client.container.ClientConsumer.getMessage(long) @bci=71, line=866 (Interpreted frame)

Thread 13808: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.jms.client.container.ClientConsumer.getMessage(long) @bci=71, line=866 (Interpreted frame)

Thread 13807: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.bootstrap.AbstractServerImpl$LifeThread.run() @bci=11, line=866 (Interpreted frame)

Thread 13806: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13805: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13804: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13803: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13802: (state = BLOCKED)
 - java.util.Arrays.copyOfRange(char[], int, int) @bci=40, line=3209 (Compiled frame)
 - java.lang.String.<init>(char[], int, int) @bci=65, line=216 (Compiled frame)
 - java.lang.StringBuffer.toString() @bci=13, line=585 (Compiled frame)
 - org.apache.log4j.PatternLayout.format(org.apache.log4j.spi.LoggingEvent) @bci=68, line=506 (Compiled frame)

Thread 13784: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)
 - org.jboss.el.util.ReferenceCache$ReferenceQueueRunner.run() @bci=1, line=159 (Interpreted frame)

Thread 13753: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.jboss.ejb3.cache.simple.SimpleStatefulCache$SessionTimeoutTask.block() @bci=11, line=210 (Interpreted frame)

Thread 13752: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - org.jboss.resource.connectionmanager.IdleRemover$IdleRemoverRunnable.run() @bci=31, line=167 (Interpreted frame)

Thread 13749: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.remoting.transport.socket.ServerThread.run() @bci=209, line=284 (Interpreted frame)

Thread 13734: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13701: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13700: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.jboss.console.plugins.AOPLister$RefreshPoller.run() @bci=41, line=898 (Interpreted frame)

Thread 13699: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() @bci=22, line=1590 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=619 (Interpreted frame)


Thread 13698: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13678: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=198 (Interpreted frame)

Thread 13554: (state = BLOCKED)

Thread 13560: (state = BLOCKED)

Thread 13559: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=132 (Compiled frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159 (Compiled frame)


Thread 13558: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Compiled frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116 (Compiled frame)

回答by Tim Bender

One possibility is that you ran out of heap space. If you run out of heap space the JVM will start to burn up the CPU at a tremendous rate doing frantic garbage collection.

一种可能性是您用完了堆空间。如果堆空间用完,JVM 将开始以惊人的速度消耗 CPU,进行疯狂的垃圾收集。

回答by Zan Lynx

Another possibility is a disk or RAM error. These are very rare especially if you are using ECC RAM, SAS or SCSI which contain error correction on data transfer. But it could happen!

另一种可能性是磁盘或 RAM 错误。这些非常罕见,尤其是当您使用包含数据传输错误纠正的 ECC RAM、SAS 或 SCSI 时。但它可能发生!

I've read about some really weird bugs seen by IBM and Sun kernel engineers doing support for big enterprises that they can only blame on stuff like this.

我读过一些非常奇怪的错误,IBM 和 Sun 内核工程师为大型企业提供支持,他们只能将这些错误归咎于此类。

This doesn't help you much, other than letting you know that you may never discover the real reason it happened and it may never happen again for years. :)

这对您没有多大帮助,只是让您知道您可能永远不会发现它发生的真正原因,并且可能多年以后再也不会发生。:)