是否有适用于 Java 的开源堆外缓存解决方案?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/7705895/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-30 21:03:46  来源:igfitidea点击:

Is there a open-source off-heap cache solution for Java?

javacachingmemory

提问by Tema

Is there any open-source alternative for Terracotta BigMemory?

Terracotta BigMemory 是否有任何开源替代品?

Actually I didn't even manage to find any commercial alternative. I'm interested in pure Java solution which will work inside JVM without any JNI and C-backed solution.

实际上我什至没有设法找到任何商业替代品。我对纯 Java 解决方案感兴趣,它可以在没有任何 JNI 和 C 支持的解决方案的情况下在 JVM 中工作。

回答by Majid Azimi

There is a very good cache solution named MapDB(JDBM4 formerly). It supports HashMapand TreeMapBut it is only application embedded. It also support persistent file based cache.

有一个非常好的缓存解决方案,名为MapDB(以前是 JDBM4)。它支持HashMapTreeMap但它只是嵌入的应用程序。它还支持基于持久文件的缓存。

Example for off heap cache:

堆外缓存示例:

DB db = DBMaker.newDirectMemoryDB().make();
ConcurrentNavigableMap<Integer, String> map = db.getTreeMap("MyCache");

Or persistent file based cache:

或基于持久文件的缓存:

DB db = DBMaker.newFileDB(new File("/home/collection.db")).closeOnJvmShutdown().make();
ConcurrentNavigableMap<Integer,String> map = db.getTreeMap("MyCache");

回答by Daniela

I've been having this question myself so I'm just going to update the previous answers with my findings.

我自己一直有这个问题,所以我将用我的发现更新以前的答案。

I found this thread from quora which also talks about the same question:

我从 quora 找到了这个帖子,它也讨论了同样的问题:

http://www.quora.com/JVM/Whats-the-best-open-source-solution-for-java-off-heap-cache

http://www.quora.com/JVM/Whats-the-best-open-source-solution-for-java-off-heap-cache

The different solution that seem to be a good fit, besides the directmemory (which has not really been updated in the last year) are

除了直接内存(去年没有真正更新)之外,似乎很合适的不同解决方案是

  • MapDB - this seems to be a very complete solution that does much more then off-heap caching and supports a lot of features
  • HugeCollections - This seems to be much less complex application then MapDB, which is focused on allocating off-heap data by extending ConcurrentMap and Map. A fork project from this, meant to target Java 8, is Chronicle-Map. A nice article about this is http://blog.shinetech.com/2014/08/26/using-hugecollections-to-manage-big-data/
  • SpyMemcached - this is a very simple single-threaded implementation with good reputation on github.
  • xmemcached - this also has a fair reputation on github but it doesn't seem to be very talked about.
  • Fast serialization - also focused on reimplementing Java Serialization with focus on off-heap usage of memory - http://ruedigermoeller.github.io/fast-serialization/
  • MapDB - 这似乎是一个非常完整的解决方案,它比堆外缓存做得更多,并支持许多功能
  • HugeCollections - 这似乎比 MapDB 复杂得多,它专注于通过扩展 ConcurrentMap 和 Map 来分配堆外数据。Chronicle-Map 是一个针对 Java 8 的 fork 项目。一篇关于此的好文章是 http://blog.shinetech.com/2014/08/26/using-hugecollections-to-manage-big-data/
  • SpyMemcached - 这是一个非常简单的单线程实现,在 github 上享有良好的声誉。
  • xmemcached - 这在 github 上也享有盛誉,但似乎没有被广泛讨论。
  • 快速序列化 - 还专注于重新实现 Java 序列化,专注于内存的堆外使用 - http://rudigermoeller.github.io/fast-serialization/

However, I would be interested furthermore to find a big enough application that is using any of these three: directmemory, SpyMemcached, xmemcached. Should I find one I will update this answer.

但是,我还有兴趣找到一个足够大的应用程序,它使用以下三个中的任何一个:directmemory、SpyMemcached、xmemcached。我应该找到一个我会更新这个答案。

回答by Peter Lawrey

I am developing a solution to be much faster, but I wouldn't suggest you use it just yet as it just a proof of concept at this stage.

我正在开发一个更快的解决方案,但我不建议您使用它,因为它只是现阶段的概念证明。

http://vanillajava.blogspot.com/2011/09/new-contributors-to-hugecollections.html

http://vanillajava.blogspot.com/2011/09/new-contributors-to-hugecollections.html

However if you have a specific requirement, it may be easier to code it yourself, to use direct ByteBuffers or memory mapped files.

但是,如果您有特定要求,自己编写代码、使用直接 ByteBuffers 或内存映射文件可能会更容易。

e.g.

例如

// using native order speeds access for values longer than a byte.
ByteBuffer bb = ByteBuffer.allocateDirect(1024*1024*1024).order(ByteOrder.nativeOrder());
// start at some location.
bb.position(0);
bb.put((byte) 1);
bb.putInt(myInt);
bb.putDouble(myDouble);

// to read back.
bb.position(0);
byte b = bb.get();
int i = bb.getInt();
double d = bb.getDouble();

You can do similarly for memory mapped files. Memory mapped files don't count towards you direct memory limit and don't use up swap space.

您可以对内存映射文件执行类似的操作。内存映射文件不计入直接内存限制,也不会占用交换空间。

Are you sure BigMemory won't do the job for you?

您确定 BigMemory 不会为您完成这项工作吗?

回答by BillMan

Looks like there is a proposal at apache:

看起来 apache 有一个提议:

http://wiki.apache.org/incubator/DirectMemoryProposal

http://wiki.apache.org/incubator/DirectMemoryProposal

回答by Jim Bethancourt

Although it isn't a solution, a guide to how to make use of ByteBuffers for your use case has been written about by Keith Gregory. Take a look at http://www.kdgregory.com/programming/java/ByteBuffer_JUG_Presentation.pdffor an overview and http://www.kdgregory.com/index.php?page=java.byteBufferfor the nitty-gritty details.

尽管这不是解决方案,但 Keith Gregory 已编写了有关如何将 ByteBuffers 用于您的用例的指南。看看http://www.kdgregory.com/programming/java/ByteBuffer_JUG_Presentation.pdf的概述和http://www.kdgregory.com/index.php?page=java.byteBuffer的细节细节.

回答by Matthias Broecheler

This implementation of a java off-heap cache uses direct memory and provides good performance in a light-weight java library:

这个 java 堆外缓存的实现使用直接内存并在轻量级 java 库中提供良好的性能:

https://github.com/snazy/ohc

https://github.com/snazy/ohc

Take a look at the benchmarking section for performance numbers. It's licensed under Apache 2.

查看性能数据的基准测试部分。它在 Apache 2 下获得许可。