为什么我的 Java 堆转储大小比已用内存小得多？

Why is my Java heap dump size much smaller than used memory?

提问by Thermometer

Problem

问题

We are trying to find the culprit of a big memory leak in our web application. We have pretty limited experience with finding a memory leak, but we found out how to make a java heap dump using jmapand analyze it in Eclipse MAT.

我们正试图找到我们的 Web 应用程序中大内存泄漏的罪魁祸首。我们在查找内存泄漏方面的经验非常有限，但我们发现了如何使用jmapEclipse MAT 进行Java 堆转储并对其进行分析。

However, with our application using 56/60GB memory, the heap dump is only 16GB in size and is even less in Eclipse MAT.

然而，我们的应用程序使用 56/60GB 内存，堆转储的大小只有 16GB，在 Eclipse MAT 中甚至更少。

Context

语境

Our server uses Wildfly 8.2.0 on Ubuntu 14.04 for our java application, whose process uses 95% of the available memory. When making the dump, our buffers/cache used space was at 56GB.

我们的服务器在 Ubuntu 14.04 上使用 Wildfly 8.2.0 作为我们的 java 应用程序，它的进程使用了​​ 95% 的可用内存。进行转储时，我们的缓冲区/缓存使用空间为 56GB。

We used the following command to create the dump: sudo -u {application user} jmap -dump:file=/mnt/heapdump/dump_prd.bin {pid}

我们使用以下命令创建转储： sudo -u {application user} jmap -dump:file=/mnt/heapdump/dump_prd.bin {pid}

The heap dump file size is 16,4GB and when analyzing it with Eclipse MAT, it says there are around 1GB live objects and ~14,8GB unreachable/shallow heap.

堆转储文件大小为 16.4GB，当使用 Eclipse MAT 对其进行分析时，它表示大约有 1GB 的活动对象和约 14.8GB 的​​不可达/浅堆。

EDIT:Here is some more info about the problem we see happening. We monitor our memory usage, and we see it grow and grow, until there is ~300mb free memory left. Then it stays around that amount of memory, until the process crashes, unfortunately without error in the application log.

编辑：这里有一些关于我们看到的问题的更多信息。我们监控我们的内存使用情况，我们看到它不断增长，直到剩下大约 300 mb 的可用内存。然后它保持在那个内存量附近，直到进程崩溃，不幸的是应用程序日志中没有错误。

This makes us assume it is a hard OOM error because this only happens when the memory is near-depleted. We use the settings -Xms25000m -Xmx40000mfor our JVM.

这让我们假设这是一个硬 OOM 错误，因为这只会在内存接近耗尽时发生。我们使用-Xms25000m -Xmx40000mJVM的设置。

Question

问题

Basically, we are wondering why the majority of our memory isn't captured in this dump. The top retained size classes don't look too suspicious, so we are wondering if there is something heap dump-related what we are doing wrong.

基本上，我们想知道为什么我们的大部分内存没有在这个转储中被捕获。顶部保留的大小类看起来不太可疑，所以我们想知道是否有一些与堆转储相关的东西我们做错了什么。

回答by Brandon

When dumping its heap, the JVM will first run a garbage collection cycle to free any unreachable objects.

转储其堆时，JVM 将首先运行垃圾收集周期以释放任何无法访问的对象。

How can I take a heap dump on Java 5 without garbage collecting first?

如何在没有垃圾收集的情况下在 Java 5 上进行堆转储？

In my experience, in a true OutOfMemoryError where your application is simply demanding more heap space than is available, this GC is a fool's errand and the final heap dump will be the size of the max. heap size.

根据我的经验，在真正的 OutOfMemoryError 中，您的应用程序只是需要比可用空间更多的堆空间，这个 GC 是一个傻瓜的差事，最终的堆转储将是最大的大小。堆大小。

When the heap dump is much smaller, that means the system was not truly out of memory, but perhaps had memory pressure. For example, there is the java.lang.OutOfMemoryError: GC overhead limit exceedederror, which means that the JVM may have been able to free enough memory to service some new allocation request, but it had to spend too much time collecting garbage.

当堆转储小得多时，这意味着系统并不是真的内存不足，而是可能有内存压力。例如，有java.lang.OutOfMemoryError: GC overhead limit exceeded错误，这意味着 JVM 可能已经能够释放足够的内存来服务一些新的分配请求，但它不得不花费太多时间收集垃圾。

It's also possible that you don't have a memory problem. What makes you think you do? You didn't mention anything about heap usage or an OutOfMemoryError. You've only mentioned the JVM's memory footprint on the operating system.

您也可能没有内存问题。是什么让你觉得你在做什么？您没有提到有关堆使用或 OutOfMemoryError 的任何内容。您只提到了 JVM 在操作系统上的内存占用。

回答by jcoll

In my experience, having a heap dump much smaller than the real memory used can be due to a leak in the JNI.

根据我的经验，堆转储比实际使用的内存小得多可能是由于 JNI 中的泄漏。

Despite you don't use directly any native code, there are certain libraries that use it to speed up.

尽管您不直接使用任何本机代码，但某些库会使用它来加快速度。

In our case, it was a Deflaterand Inflaternot properly ended.

在我们的例子中，它是一个Deflater和Inflater没有正确结束。