如何从在java中使用FileChannel映射的内存中取消映射文件？

How to unmap a file from memory mapped using FileChannel in java?

提问by learner135

I am mapping a file("sample.txt") to memory using FileChannel.map()and then closing the channel using fc.close(). After this when I write to the file using FileOutputStream, I am getting the following error:

我正在将文件（“sample.txt”）映射到内存FileChannel.map()，然后使用fc.close(). 在此之后，当我使用 FileOutputStream 写入文件时，出现以下错误：

java.io.FileNotFoundException: sample.txt (The requested operation cannot be per formed on a file with a user-mapped section open)

java.io.FileNotFoundException: sample.txt（无法对打开用户映射部分的文件执行请求的操作）

File f = new File("sample.txt");
RandomAccessFile raf = new RandomAccessFile(f,"rw");
FileChannel fc = raf.getChannel();
MappedByteBuffer mbf = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
fc.close();
raf.close();

FileOutputStream fos = new FileOutputStream(f);
fos.write(str.getBytes());
fos.close();

I presume this may be due to file being still mapped to the memory even after I close the FileChannel. Am I right?. If so, how can I "unmap" the file from memory?(I can't find any methods for this in the API). Thanks.

我认为这可能是因为即使在我关闭FileChannel. 我对吗？。如果是这样，我如何从内存中“取消映射”文件？（我在 API 中找不到任何方法）。谢谢。

Edit: Looks like it(adding an unmap method) was submitted as RFE to sun some time back: http://bugs.sun.com/view_bug.do?bug_id=4724038

编辑：看起来它（添加一个取消映射方法）作为 RFE 提交给 sun 一段时间：http: //bugs.sun.com/view_bug.do?bug_id=4724038

采纳答案by Edward Dale

From the MappedByteBufferjavadoc:

从MappedByteBufferjavadoc：

A mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is garbage-collected.

映射的字节缓冲区及其表示的文件映射在缓冲区本身被垃圾收集之前一直有效。

Try calling System.gc()? Even that's only a suggestion to the VM.

试试打电话System.gc()？即使这只是对 VM 的建议。

回答by BenM

The mapped memory is used until it is freed by the garbage collector.

映射的内存会一直使用，直到被垃圾收集器释放为止。

From FileChanneldocs

来自FileChannel文档

A mapping, once established, is not dependent upon the file channel that was used to create it. Closing the channel, in particular, has no effect upon the validity of the mapping.

映射一旦建立，就不再依赖于用于创建它的文件通道。特别是关闭通道对映射的有效性没有影响。

From MappedByteBufferjava doc

来自MappedByteBufferjava doc

A mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is garbage-collected.

映射的字节缓冲区及其表示的文件映射在缓冲区本身被垃圾收集之前一直有效。

So I would suggest ensuring there are no remaining references to the mapped byte buffer and then requesting a garbage collection.

所以我建议确保没有对映射字节缓冲区的剩余引用，然后请求垃圾收集。

回答by Timur Yusupov

Following static method could be used:

可以使用以下静态方法：

public static void unmap(MappedByteBuffer buffer)
{
   sun.misc.Cleaner cleaner = ((DirectBuffer) buffer).cleaner();
   cleaner.clean();
}

But this is unsafe solution because of following:
1) Lead to failures if someone use MappedByteBuffer after unmap
2) It relies on MappedByteBuffer implementation details

但这是不安全的解决方案，因为以下原因：
1）如果有人在取消映射后使用 MappedByteBuffer 会导致失败
2）它依赖于 MappedByteBuffer 实现细节

回答by Mr Ed

To work around this bug in Java, I had to do the following, which will work ok for small to medium-sized files:

要解决 Java 中的此错误，我必须执行以下操作，这对于中小型文件可以正常工作：

    // first open the file for random access
    RandomAccessFile raf = new RandomAccessFile(file, "r");

    // extract a file channel
    FileChannel channel = raf.getChannel();

    // you can memory-map a byte-buffer, but it keeps the file locked
    //ByteBuffer buf =
    //        channel.map(FileChannel.MapMode.READ_ONLY, 0, channel.size());

    // or, since map locks the file... just read the whole file into memory
    ByteBuffer buf = ByteBuffer.allocate((int)file.length());
    int read = channel.read(buf);

    // .... do something with buf

    channel.force(false);  // doesn't help
    channel.close();       // doesn't help
    channel = null;        // doesn't help
    buf = null;            // doesn't help
    raf.close();           // try to make sure that this thing is closed!!!!!

回答by somedude

If the mapped file buffer object can be guaranteed to be eligible for garbage collection, you don't need to GC the whole VM to get the buffer's mapped memory to be released. You can call System.runFinalization() . This will call the finalize() method on the mapped file buffer object (if it there are no references to it in your app threads) which will release the mapped memory.

如果可以保证映射文件缓冲区对象符合垃圾回收条件，则不需要对整个 VM 进行 GC 来释放缓冲区的映射内存。您可以调用 System.runFinalization() 。这将调用映射文件缓冲区对象上的 finalize() 方法（如果应用程序线程中没有对它的引用），这将释放映射内存。

回答by kittylyst

The correct solution here is to use try-with-resources.

这里的正确解决方案是使用 try-with-resources。

This allows the creation of the Channel & the other resources to be scoped to a block. Once the block exits, the Channel & other resources are gone & subsequently cannot be used (as nothing has a reference to them).

这允许将 Channel 和其他资源的创建范围限定为一个块。一旦块退出，通道和其他资源就消失了，随后无法使用（因为没有任何东西可以引用它们）。

The memory-mapping still won't be undone until the next GC run, but at least there aren't any dangling references to it.

内存映射在下一次 GC 运行之前仍然不会被撤消，但至少没有任何悬空引用。

回答by Whome

[WinXP,SunJDK1.6] I had a mapped ByteBuffer taken from filechannel. After reading SO posts finally managed to call a cleaner through reflection without any sun.* package imports. No longer file lock is lingering.

[WinXP,SunJDK1.6] 我有一个从文件通道中获取的映射 ByteBuffer。在阅读了 SO 帖子后，终于设法通过反射调用了一个清洁器，而没有任何 sun.* 包导入。文件锁不再挥之不去。

editAdded JDK9+ code(Luke Hutchison).

编辑添加了 JDK9+ 代码（Luke Hutchison）。

private static void closeDirectBuffer(ByteBuffer cb) {
    if (cb==null || !cb.isDirect()) return;
    // we could use this type cast and call functions without reflection code,
    // but static import from sun.* package is risky for non-SUN virtual machine.
    //try { ((sun.nio.ch.DirectBuffer)cb).cleaner().clean(); } catch (Exception ex) { }

    // JavaSpecVer: 1.6, 1.7, 1.8, 9, 10
    boolean isOldJDK = System.getProperty("java.specification.version","99").startsWith("1.");  
    try {
        if (isOldJDK) {
            Method cleaner = cb.getClass().getMethod("cleaner");
            cleaner.setAccessible(true);
            Method clean = Class.forName("sun.misc.Cleaner").getMethod("clean");
            clean.setAccessible(true);
            clean.invoke(cleaner.invoke(cb));
        } else {
            Class unsafeClass;
            try {
                unsafeClass = Class.forName("sun.misc.Unsafe");
            } catch(Exception ex) {
                // jdk.internal.misc.Unsafe doesn't yet have an invokeCleaner() method,
                // but that method should be added if sun.misc.Unsafe is removed.
                unsafeClass = Class.forName("jdk.internal.misc.Unsafe");
            }
            Method clean = unsafeClass.getMethod("invokeCleaner", ByteBuffer.class);
            clean.setAccessible(true);
            Field theUnsafeField = unsafeClass.getDeclaredField("theUnsafe");
            theUnsafeField.setAccessible(true);
            Object theUnsafe = theUnsafeField.get(null);
            clean.invoke(theUnsafe, cb);
        }
    } catch(Exception ex) { }
    cb = null;
}

Ideas were taken from these posts.
* How to unmap a file from memory mapped using FileChannel in java?
* Examples of forcing freeing of native memory direct ByteBuffer has allocated, using sun.misc.Unsafe?
* https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/apache/lucene/store/bytebuffer/ByteBufferAllocator.java#L40

想法来自这些帖子。
*如何从在java 中使用FileChannel 映射的内存中取消映射文件？
*使用sun.misc.Unsafe 强制释放本机内存直接ByteBuffer 已分配的示例？
* https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/apache/lucene/store/bytebuffer/ByteBufferAllocator.java#L40

回答by Dmytro Voloshyn

sun.misc.Cleaner javadoc says:

sun.misc.Cleaner javadoc 说：

General-purpose phantom-reference-based cleaners. Cleaners are a lightweight and more robust alternative to finalization. They are lightweight because they are not created by the VM and thus do not require a JNI upcall to be created, and because their cleanup code is invoked directly by the reference-handler thread rather than by the finalizer thread. They are more robust because they use phantom references, the weakest type of reference object, thereby avoiding the nasty ordering problems inherent to finalization. A cleaner tracks a referent object and encapsulates a thunk of arbitrary cleanup code. Some time after the GC detects that a cleaner's referent has become phantom-reachable, the reference-handler thread will run the cleaner. Cleaners may also be invoked directly; they are thread safe and ensure that they run their thunks at most once. Cleaners are not a replacement for finalization. They should be used only when the cleanup code is extremely simple and straightforward. Nontrivial cleaners are inadvisable since they risk blocking the reference-handler thread and delaying further cleanup and finalization.

通用基于幻象参考的清洁器。清洁器是一种轻量级且更强大的定型替代方案。它们是轻量级的，因为它们不是由 VM 创建的，因此不需要创建 JNI 向上调用，并且因为它们的清理代码由引用处理程序线程而不是终结器线程直接调用。它们更加健壮，因为它们使用幻像引用（最弱的引用对象类型），从而避免了终结固有的令人讨厌的排序问题。清洁器跟踪所指对象并封装大量任意清理代码。在 GC 检测到清洁器的引用对象变为幻影可达一段时间后，引用处理程序线程将运行清洁器。也可以直接调用清洁器；它们是线程安全的，并确保它们最多运行一次 thunk。清洁器不是最终确定的替代品。只有当清理代码非常简单和直接时才应该使用它们。非平凡的清理器是不可取的，因为它们有阻塞引用处理程序线程并延迟进一步清理和完成的风险。

Running System.gc() is acceptable solution if your buffers total size is small, but if I was mapping gigabytes of files I would try to implement like this:

如果您的缓冲区总大小很小，运行 System.gc() 是可以接受的解决方案，但如果我要映射千兆字节的文件，我会尝试像这样实现：

((DirectBuffer) buffer).cleaner().clean()

But! Make sure you don't access that buffer after cleaning or you will end up with:

但！确保在清理后不要访问该缓冲区，否则最终会得到：

A fatal error has been detected by the Java Runtime Environment: EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000002bcf700, pid=7592, tid=10184 JRE version: Java(TM) SE Runtime Environment (8.0_40-b25) (build 1.8.0_40-b25) Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode windows-amd64 compressed oops) Problematic frame: J 85 C2 java.nio.DirectByteBuffer.get(I)B (16 bytes) @ 0x0000000002bcf700 [0x0000000002bcf6c0+0x40] Failed to write core dump. Minidumps are not enabled by default on client versions of Windows An error report file with more information is saved as: C:\Users\?????\Programs\testApp\hs_err_pid7592.log Compiled method (c2) 42392 85 4 java.nio.DirectByteBuffer::get (16 bytes) total in heap [0x0000000002bcf590,0x0000000002bcf828] = 664 relocation [0x0000000002bcf6b0,0x0000000002bcf6c0] = 16 main code [0x0000000002bcf6c0,0x0000000002bcf760] = 160 stub code
[0x0000000002bcf760,0x0000000002bcf778] = 24 oops
[0x0000000002bcf778,0x0000000002bcf780] = 8 metadata
[0x0000000002bcf780,0x0000000002bcf798] = 24 scopes data
[0x0000000002bcf798,0x0000000002bcf7e0] = 72 scopes pcs
[0x0000000002bcf7e0,0x0000000002bcf820] = 64 dependencies
[0x0000000002bcf820,0x0000000002bcf828] = 8

Java 运行时环境检测到一个致命错误：EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000002bcf700, pid=7592, tid=10184 JRE 版本：Java(TM) SE 运行时环境 (8.0_8)-40-b40-b40 b25) Java VM：Java HotSpot(TM) 64 位服务器 VM（25.40-b25 混合模式 windows-amd64 压缩 oops）有问题的框架：J 85 C2 java.nio.DirectByteBuffer.get(I)B（16 字节）@ 0x0000000002bcf700 [0x0000000002bcf6c0+0x40] 无法写入核心转储。默认情况下，Windows 客户端版本不启用小型转储 包含更多信息的错误报告文件保存为：C:\Users\?????\Programs\testApp\hs_err_pid7592.log 编译方法 (c2) 42392 85 4 java. nio.DirectByteBuffer::get (16 bytes) 总在堆 [0x0000000002bcf590,0x0000000002bcf828] = 664 重定位 [0x0000000002bcf6b0,
[0x0000000002bcf760,0x0000000002bcf778] = 24个糟糕
[0x0000000002bcf778,0x0000000002bcf780] = 8元数据
[0x0000000002bcf780,0x0000000002bcf798] = 24个范围数据
[0x0000000002bcf798,0x0000000002bcf7e0] = 72范围件
[0x0000000002bcf7e0,0x0000000002bcf820] = 64依赖性
[0x0000000002bcf820,0x0000000002bcf828] = 8

Good luck!

祝你好运！

回答by u7590921

I found out information about unmap, it is a method of FileChannelImpland not accessible, so you can invoke it by java reflect like:

我发现了关于 的信息unmap，它是一种方法FileChannelImpl并且不可访问，因此您可以通过 java reflect 调用它，例如：

public static void unMapBuffer(MappedByteBuffer buffer, Class channelClass) {
    if (buffer == null) {
        return;
    }

    try {
        Method unmap = channelClass.getDeclaredMethod("unmap", MappedByteBuffer.class);
        unmap.setAccessible(true);
        unmap.invoke(channelClass, buffer);
    } catch (NoSuchMethodException e) {
        e.printStackTrace();
    } catch (IllegalAccessException e) {
        e.printStackTrace();
    } catch (InvocationTargetException e) {
        e.printStackTrace();
    }
}

回答by Droid Teahouse

It is funny to see so many recommendations to do what Item 7 in 'Effective Java' specifically says not to do. A termination method like what @Whome did and no references to the buffer is what is needed. GC cannot be forced. But that doesn't stop developers from trying. Another workaround I found was to use WeakReferences from http://jan.baresovi.cz/dr/en/java#memoryMap

看到如此多的建议去做“Effective Java”中第 7 条明确规定不要做的事情，这很有趣。需要一种终止方法，例如@Whome 所做的并且没有对缓冲区的引用。GC 不能强制。但这并不能阻止开发人员尝试。我发现的另一个解决方法是使用来自http://jan.baresovi.cz/dr/en/java#memoryMap 的WeakReferences

final MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, size);
....
final WeakReference<mappedbytebuffer> bufferWeakRef = new WeakReference<mappedbytebuffer>(bb);
bb = null;

final long startTime = System.currentTimeMillis();
while(null != bufferWeakRef.get()) {
  if(System.currentTimeMillis() - startTime > 10)
// give up
    return;
    System.gc();
    Thread.yield();
}