java嵌入式库磁盘键值数据库

java embedded library on-disk key-value database

提问by Groostav

What I thinkI'm looking for is a no-SQL, library-embedded, on disk (ie not in-memory) database, thats accessible from java (and preferably runs inside my instance of the JVM). That's not really much of a database, and I'm tempted to roll-my-own. Basically I'm looking for the "should we keep this in memory or put it on disk" portion of a database.

我想我正在寻找的是一个无 SQL、嵌入库的磁盘（即非内存）数据库，可以从 java 访问（最好在我的 JVM 实例中运行）。这不是一个真正的数据库，我很想自己动手。基本上，我正在寻找数据库的“我们应该将其保存在内存中还是将其放在磁盘上”部分。

Our model has grown to several gigabytes. Right now this is all done in memory, meaning we're pushing the JVM for upward of several gigabytes. It's currently all stored in a flat XML file, serialized and deserialized with xstream and compressed with Java'a built in gzip libraries. That's worked well when our model stays under 100MB, but now that its larger than that its becoming a problem.

我们的模型已经增长到几 GB。现在这一切都是在内存中完成的，这意味着我们正在推动 JVM 增加几 GB。它目前全部存储在一个平面 XML 文件中，使用 xstream 进行序列化和反序列化，并使用 Java'a 内置的 gzip 库进行压缩。当我们的模型保持在 100MB 以下时，这很有效，但现在它大于 100MB 就成了一个问题。

loosely speaking that model can be broken down as

松散地说，该模型可以分解为

• Project
  1. configuration component (directed-acyclic-graph), not at all database friendly
  2. a list of a dozen "experiment" structures
     • each containing a list of about a dozen "run-model" structures.
       1. each run-model contains hundreds of megs of data. Once written they are never edited.

• 项目
  1. 配置组件（有向无环图），完全不适合数据库
  2. 十几个“实验”结构的清单
     • 每个包含大约十几个“运行模型”结构的列表。
       1. 每个运行模型都包含数百兆的数据。一旦编写，它们就永远不会被编辑。

What I'd like to do is have something that conforms to a map interface, of guid -> run-model. This mini-database would keep a flat table of these objects. On our experiment model, we would replace the list of run-models with a list of guids, and add, at the application layer, a get call to this map, which would pull it off the disk and into memory.

我想要做的是符合地图界面的东西，guid -> run-model。这个迷你数据库将保存这些对象的平面表。在我们的实验模型中，我们将使用 guid 列表替换运行模型列表，并在应用层添加对这个映射的 get 调用，这会将它从磁盘中拉到内存中。

That means we can keep configuration of our program in XML (which I'm very happy with) and keep a table of the big data in a DBMS that will keep us from consuming multi-GB of memory. On program start and exit I could then load and unload the two portions of our model (the config section in XML, and the run-models in the database format) from an archiving format.

这意味着我们可以在 XML 中保留我们的程序配置（我很满意）并在 DBMS 中保留一个大数据表，这将防止我们消耗多 GB 的内存。在程序启动和退出时，我可以从存档格式加载和卸载模型的两个部分（XML 中的配置部分，以及数据库格式中的运行模型）。

I'm sort've feeling gung-ho about this, and think that I could probably implement it with some of X-Stream's XML inspection strategies and a custom map implementation, but something a voice in the back of my head is telling me I should find a library to do it instead.

我对此感到很兴奋，并认为我可能可以使用 X-Stream 的一些 XML 检查策略和自定义地图实现来实现它，但是我脑后的声音告诉我我应该找一个图书馆来代替。

Should I roll my own or is there a database that's small enough to fit this bill?

我应该自己推出还是有一个足够小的数据库来满足这个要求？

Thanks guys,

谢谢你们，

-Geoff

-杰夫

采纳答案by cruftex

http://www.mapdb.org/

http://www.mapdb.org/

Also take a look at this question: Alternative to BerkeleyDB?

也看看这个问题：Alternative to BerkeleyDB?

回答by leventov

Since MapDB is a possible solution for your problem, Chronicle Mapis also worth consideration. It's an embeddable Java key-value store, optionally persistent, offering a very similar programming model to MapDB: it also via the vanilla java.util.Mapinterface and transparent serialization of keys and values.

由于 MapDB 是您问题的可能解决方案，因此Chronicle Map也值得考虑。它是一个可嵌入的 Java 键值存储，可选择持久化，提供与 MapDB 非常相似的编程模型：它还通过普通java.util.Map接口和键和值的透明序列化。

The major difference is that according to third-party benchmarks, Chronicle Map is times faster than MapDB.

主要区别在于，根据第三方基准测试，Chronicle Map 比 MapDB 快几倍。

Regarding stability, no bugs were reported about the Chronicle Map data storage for months now, while it is in active use in many projects.

关于稳定性，几个月来没有报告关于 Chronicle Map 数据存储的错误，而它在许多项目中都被积极使用。

Disclaimer: I'm the developer of Chronicle Map.

免责声明：我是 Chronicle Map 的开发者。