java 无论内存需求有多大，有没有办法强制JVM使用交换？

Is there any way to force a JVM to use swap no matter how big the memory requirement is?

提问by mahonya

Here is my situation: I have a task at hand that requires lots of memory. I do not have enough ram, and no matter what I tried (Jrockit with /3gb switch etc), I can't give JVM enough ram and the operation is terminated with an exception, telling me I need more heap space.

这是我的情况：我手头有一项需要大量内存的任务。我没有足够的内存，无论我尝试过什么（带有 /3gb 开关的 Jrockit 等），我都无法为 JVM 提供足够的内存，并且操作因异常而终止，告诉我我需要更多堆空间。

Is there any way I can force the JVM to use the OS's swapping mechanism so that it won't run out of memory? This is Windows xp 32 bit

有什么办法可以强制 JVM 使用操作系统的交换机制，以免内存不足？这是 Windows xp 32 位

It would take ages, but I would not care, I just need this operation to be completed.

这需要很长时间，但我不在乎，我只需要完成这个操作。

I've run out of options, and I have no control over any of the variables here..

我已经没有选择了，我无法控制这里的任何变量..

This is a required edit, since I am having the same response from pretty much everyone :) This is not my code. Someone has written a tool that reads an xml file into a repository. The tool uses EMF, and loads the whole model at once. All I can do is to feed it the XML file. In case of native code running under Windows or Linux etc, the OS provides memory to it, using virtual memory/swap space, and the app does not know about it. I was wondering if it is possible to do the same with the JVM. Under Windows 32 bit, -Xmx can go up to a certain amount, but that is not enough. Going out and buying new hardware is not an option for me for the moment. So I was wondering if it is possible to make the JVM work like native processes. Slow, but still working. Apparently that is not possible, and I am out of luck. I just need to know if I'm really out of options.

这是必需的编辑，因为我得到了几乎所有人的相同回应:) 这不是我的代码。有人编写了一个将 xml 文件读入存储库的工具。该工具使用 EMF，并一次加载整个模型。我所能做的就是将 XML 文件提供给它。对于在 Windows 或 Linux 等下运行的本机代码，操作系统为其提供内存，使用虚拟内存/交换空间，而应用程序并不知道它。我想知道是否可以对 JVM 做同样的事情。在 Windows 32 位下，-Xmx 可以上升到一定数量，但这还不够。外出购买新硬件目前对我来说不是一种选择。所以我想知道是否有可能让 JVM 像本地进程一样工作。慢，但仍在工作。显然这是不可能的，而且我很不走运。我只需要知道我是否

采纳答案by rodion

Apparently there is one way around the limits of Java heap. It is even used in a commercial product called BigMemorywhich basically allows you to have almost unlimited memory by transparently swapping out to OS swap and/or to disk if needed.

显然有一种方法可以绕过 Java 堆的限制。它甚至用于名为BigMemory的商业产品中，它基本上允许您通过透明地交换到操作系统交换和/或磁盘（如果需要）来拥有几乎无限的内存。

The idea is to use direct ByteBuffers to store your objects data. Because direct byte buffers' contents are stored in native process memory (as opposed to heap) you can rely on OS swap mechanism to swap memory out for you. I found this on this website(search for 'direct byte buffer' on the page).

这个想法是使用 direct ByteBuffers 来存储您的对象数据。因为直接字节缓冲区的内容存储在本机进程内存（而不是堆）中，您可以依靠操作系统交换机制为您换出内存。我在这个网站上找到了这个（在页面上搜索“直接字节缓冲区”）。

Here is how you can implement it (java-pseudo-code'ish):

以下是实现它的方法（java-pseudo-code'ish）：

class NativeMemoryCache{
  private Map<Object, ByteBuffer> data = new HashMap<...>();

  public void put(Object key, Serializable object){
    byte[] bytes = serialize(object);
    //allocate native memory to store our object
    ByteBuffer buf = ByteBuffer.allocateDirect(bytes.length);
    buf.put(bytes);
    buf.flip();
    data.put(key, buf);
  }

  public Object get(Object key){
    ByteBuffer buf = data.get(key).duplicate();
    byte[] bytes = new byte[buf.remaining()];
    buf.get(bytes);
    return deserialize(bytes);
  }

  private byte[] serialize(Object obj){ ... }
  private Object deserialize(byte[] bytes){ ... }
}

Hope you get the idea. You just need to implement the serialization (you can also compress your objects using zip. This will be effective if you have few big objects especially ones containing zippable data like strings).

希望你能明白。您只需要实现序列化（您也可以使用 zip 压缩您的对象。如果您的大对象很少，尤其是包含字符串等可压缩数据的对象，这将是有效的）。

Of course NativeMemoryCacheobject, datahash map and keys will be in heap, but that should not take much memory.

当然NativeMemoryCacheobject、datahash map 和keys 将在堆中，但这不应该占用太多内存。

回答by sleske

As pointed out by the other answers, you use the -Xmxswitch to give more RAM to the JVM.

正如其他答案所指出的，您使用-Xmx开关为 JVM 提供更多 RAM。

However, there's a limit on how high you can go. On a 32bit system, this will probably be 2GiB, maybe 3 or 4 GiB if the JVM supports it. For the Sun JVM, the limit is 1500MiB on 32bit Windows, according to Java -Xmx, Max memory on system.

但是，您可以走多高是有限制的。在 32 位系统上，这可能是 2GiB，如果 JVM 支持，可能是 3 或 4 GiB。根据Java -Xmx, Max memory on system ，对于 Sun JVM，32 位 Windows 上的限制为 1500MiB 。

For fundamental architectural reasons, a process cannot (without special techniques) get more than 4 GiB of memory (includingany swap space it may use), that's why the limit on -Xmxvalues exists.

由于基本的架构原因，进程不能（没有特殊技术）获得超过 4 GiB 的内存（包括它可能使用的任何交换空间），这就是-Xmx存在值限制的原因。

If you have tried the maximum possible value, and still get OOM Errors, then your only options are:

如果您尝试了最大可能值，但仍然出现 OOM 错误，那么您唯一的选择是：

• fix the application so it needs less RAM

• 修复应用程序，使其需要更少的 RAM

or

或者

• move it to a 64bit OS, and increase -Xmxeven further

• 将其移至 64 位操作系统，并-Xmx进一步增加

Edit:

编辑：

Note that the 4 GiB limit is a limitation of the CPU architecture, so it applies to any process, Java or not. So even native allocation tricks won't help you here. The only way around it is to use more than one process, but that would require a fundamental rewrite of the application, which would probably be as complicated as just fixing the app to use less RAM. So the two options above are your only (sensible) options.

请注意，4 GiB 限制是CPU 架构的限制，因此它适用于任何进程，无论是否为 Java。因此，即使是本机分配技巧在这里也无济于事。解决它的唯一方法是使用多个进程，但这需要对应用程序进行根本性的重写，这可能与修复应用程序以使用更少的 RAM 一样复杂。所以上面的两个选项是你唯一的（明智的）选项。

Edit 2:

编辑2：

To address the new part of your question:

要解决问题的新部分：

I was wondering if it is possible to make the JVM work like native processes.

我想知道是否有可能使 JVM 像本机进程一样工作。

This is a misunderstanding. The JVM doeswork like native process in this respect: The heap it uses is located in memory allocated from the OS by the JVM; to the OS this is just allocated memory, and the OS will swap it out like any other memory if it feels like it - there is nothing special about this.

这是一种误解。在这方面，JVM确实像本地进程一样工作：它使用的堆位于 JVM 从操作系统分配的内存中；对于操作系统来说，这只是分配的内存，如果感觉合适，操作系统会像任何其他内存一样将其换出 - 这没有什么特别之处。

The reason that the heap cannot grow indefinitely is not that it cannot be larger than physical RAM (it can, I have tried it at least on Linux/x86), but that each OS process (which the JVM is) cannot get more than 4GiB RAM. So on 32bit systems you can never have more than 4GiB heap. In practice, it may be much less because the heap memory must not be fragmented (see e.g. Java maximum memory on Windows XP), but the 4GiB is a hard, unavoidable limit.

堆不能无限增长的原因不是它不能大于物理 RAM（它可以，我至少在 Linux/x86 上尝试过），而是每个 OS 进程（JVM 是）不能超过 4GiB内存。因此，在 32 位系统上，您永远不会拥有超过 4GiB 的堆。在实践中，它可能要少得多，因为堆内存不能被碎片化（参见例如Windows XP 上的 Java 最大内存），但 4GiB 是一个难以避免的限制。

回答by AlexR

According to my experience JVM requests memory from OS that can allocate it either in RAM in swap. It depends on how much resources do you have. The memory you can allocate in java does not depend on your RAM but on command line option -Xmxyou specify when you are running your JVM. If for example it is not enough memory in RAM JVM receives it from swap and (I believe) even does not know about that.

根据我的经验，JVM 从 OS 请求内存，这些内存可以在 RAM 中的交换中分配。这取决于你有多少资源。您可以在 Java 中分配的内存不取决于您的 RAM，而是取决于-Xmx您在运行 JVM 时指定的命令行选项。例如，如果 RAM 中的内存不足，JVM 会从交换接收它，并且（我相信）甚至不知道这一点。

BTW IMHO you do not reallyneed so much memory. I am agree with guys that said that. I'd suggest you to review your design.

顺便说一句，恕我直言，你真的不需要这么多内存。我同意那些说这话的人。我建议你检查一下你的设计。

回答by Peter Lawrey

If you don't have enough RAM you need to change your code so the application fits into memory. If you make the JVM large enough that it has to swap to disk the application it will as good as hang. The heap in the JVM is not designed to run off disk.

如果您没有足够的 RAM，则需要更改代码以使应用程序适合内存。如果您使 JVM 足够大，以至于它必须将应用程序交换到磁盘，那么它就像挂起一样好。JVM 中的堆不是为在磁盘外运行而设计的。

I suspect the problem you have is that you cannot allocate enough continuous memory which is a a requirement for the JVM. As you use more of the available memory, it is harder to get a large continuous memory block with an 32-bit OS.

我怀疑您遇到的问题是您无法分配足够的连续内存，这是 JVM 的要求。随着您使用更多可用内存，在 32 位操作系统中获得大的连续内存块变得更加困难。

It is either time to get more memory, which is relatively cheap these days. or reduce your memory requirement. Using swap will just take forever to complete.

现在是获得更多内存的时候了，现在这些内存相对便宜。或减少您的内存需求。使用交换将永远完成。

BTW: You can buy a 24 GB server for about ￡1,800 and 64 GB server for about ￡4,200. For ￡53,000 you can get a server with 1 TB of memory! :D

顺便说一句：您可以花 1,800 英镑左右购买 24 GB 服务器，花 4,200 英镑左右购买 64 GB 服务器。只需 53,000 英镑，您就可以获得 1 TB 内存的服务器！:D