适用于 Node.js 的最快、非基于内存的多进程键值存储
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/21204979/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Fastest, non-memory-based, multi-process key-value store for Node.js
提问by Ruben Verborgh
What is the fastest non-memory key-value store for Node.js supporting multiple processes?
支持多进程的 Node.js 最快的非内存键值存储是什么？
I need to store simple key-value string/string pairs (not documents or JSON, just strings).
Here are some examples (there would be millions of those):
我需要存储简单的键值字符串/字符串对（不是文档或 JSON，只是字符串）。
以下是一些示例（将有数百万个）：
- 12345678– abcdefghijklmnopabcdefghijklmnop
- 86358098– ahijklmnopbcdefgahijklmnopbcdefg
- abcdefghijklmnopabcdefghijklmnop- 12345678
- ahijklmnopbcdefgahijklmnopbcdefg- 86358098
- 12345678– abcdefghijklmnopabcdefghijklmnop
- 86358098– ahijklmnopbcdefgahijklmnopbcdefg
- abcdefghijklmnopabcdefghijklmnop- 12345678
- ahijklmnopbcdefgahijklmnopbcdefg- 86358098
I have tried:
我试过了：
- Redis: it's really fast and does everything I need, but consumes too much RAM.
- LevelDB: it's fast and not too heavy on RAM, but only single-process.
A workaround for LevelDB is multilevel, which exposes a single LevelDB process though HTTP.
But that of course comes at a cost; I need something fast.
LevelDB 的解决方法是multilevel，它通过 HTTP 公开单个 LevelDB 进程。
但这当然是有代价的。我需要一些快速的东西。
Is there any key-value store that:
是否有任何键值存储：
- supports Node.js or has bindings for it;
- stores string/string pairs;
- supports multiple processes;
- does not entirely reside in memory;
- is fast?
- 支持 Node.js 或为其绑定；
- 存储字符串/字符串对；
- 支持多进程；
- 不完全驻留在内存中；
- 快吗？
I only care about reading. Fast multi-process reading is necessary, but not writing.
I'm happy with the current speed of LevelDB, just not with the fact that it is single-process.
我只关心阅读。快速的多进程读取是必要的，但不是写入。
我对 LevelDB 目前的速度很满意，只是对它是单进程的事实不满意。
Additional details:
额外细节：
- I'm talking about some 50 million key/value pairs, with keys and values between 8 and 500 chars.
- The code will run on a regular Linux server.
- Memory usage should be limited to a few gigabytes (4GB is fine, 8GB is acceptable)
- Reading will happen way more than writing; actually, I could do without writing.
- Speed is more important than anything (given memory and multi-process constraint are respected).
- 我说的是大约 5000 万个键/值对，键和值在 8 到 500 个字符之间。
- 该代码将在常规 Linux 服务器上运行。
- 内存使用量应限制在几 GB 以内（4GB 可以，8GB 也可以）
- 阅读比写作更重要；实际上，我可以不写作。
- 速度比什么都重要（考虑到内存和多进程约束）。
回答by Didier Spezia
I would suggest to have a look at LMDB(which is the most efficient engine for OpenLDAP, and used in a number of other open-source projects).
我建议看看LMDB（它是 OpenLDAP 最有效的引擎，并在许多其他开源项目中使用）。
LMDB is an embedded key/value store, with a Berkeley-DB or LevelDB like API, does not have to store everything in memory, and can support access from multiple processes. There are Node.js bindings:
LMDB 是一个嵌入式键/值存储，具有 Berkeley-DB 或 LevelDB 之类的 API，不必将所有内容都存储在内存中，并且可以支持来自多个进程的访问。有 Node.js 绑定：
回答by Polor Beer
You can try ssdb, a redis protocol compatible database built upon leveldb.
你可以试试ssdb，一个基于 leveldb 的 redis 协议兼容数据库。
https://github.com/ideawu/ssdb
https://github.com/ideawu/ssdb
You can use the existing node-redisclient, though some of the commands may vary.
您可以使用现有node-redis客户端，但某些命令可能会有所不同。
基准：
Redis (100.000x)
13,540 op/s ? set small
13,289 op/s ? set medium
13,279 op/s ? set large
13,651 op/s ? get large
13,681 op/s ? get medium
14,428 op/s ? get small
SSDB (100.000x)
12,252 op/s ? set small
11,824 op/s ? set medium
11,720 op/s ? set large
13,810 op/s ? get large
13,593 op/s ? get medium
12,696 op/s ? get small
lmdb (100.000x)
4,616 op/s ? set small
11,104 op/s ? set medium
17,283 op/s ? set large
13,778 op/s ? get large
16,002 op/s ? get medium
50,562 op/s ? get small
multilevel (100.000x)
6,124 op/s ? set small
5,900 op/s ? set medium
5,944 op/s ? set large
6,215 op/s ? get large
6,125 op/s ? get medium
6,310 op/s ? get small
As you can see, ssdbis almost as fast as redis, and it is designed for persistent storage. lmdb@didier-spezia mentioned is ultra fast for getting small data, but setting one is slow.
正如你所看到的，ssdb它几乎和 redis 一样快，而且它是为持久存储而设计的。lmdb@didier-spezia 提到的获取小数据的速度非常快，但设置一个速度很慢。
回答by Amnon
There is FaceBook's RocksDBthat is supposed to be fast (especially on SSD storage), and there are also others such as LMDB (already mentioned) and WiredTiger
有 FaceBook 的RocksDB应该很快（尤其是在 SSD 存储上），还有其他的，例如 LMDB（已经提到）和WiredTiger
You mentioned Redis - If you'd like to use the Redis API but have one of the above Key/Value databases as the storage instead of your RAM, there are two projects I know of (though haven't tested them): LedisDB(written in Go) and ardb(written in C++).
您提到了 Redis - 如果您想使用 Redis API 但将上述键/值数据库之一作为存储而不是您的 RAM，那么我知道有两个项目（尽管尚未对其进行测试）：LedisDB（用 Go 编写）和adb（用 C++ 编写）。
I've recently started testing what seems like a very promising though yet less known (though I'm sure that will change) key value database library named CuttDB. It has very fast performance and built to handle large amounts of data on the HDD. It even includes a Memcached server interface.
我最近开始测试一个看起来很有前途但鲜为人知（尽管我确信这会改变）名为CuttDB 的键值数据库库。它具有非常快的性能，专为处理 HDD 上的大量数据而构建。它甚至包括一个 Memcached 服务器接口。
回答by AlexGad
The problem you are going to run into is that "lightning fast" and disk don't mix especially if you have random access reads as you do in a key-value system. You need to get as much data into memory as possible since reading from memory is many magnitudes faster than reading from disk.
您将遇到的问题是“闪电般快速”和磁盘不会混合，特别是如果您像在键值系统中那样进行随机访问读取。您需要将尽可能多的数据放入内存，因为从内存中读取比从磁盘读取快很多数量级。
Is the reason you want to minimize memory because this will be an embedded database? If so, you might want to look at Empress - http://www.empress.com. Have used it in a couple of projects and you can configure how much gets loaded. However, its got the overhead of an RDBMS so not sure it will be as lean as you want.
你想最小化内存的原因是因为这将是一个嵌入式数据库吗？如果是这样，您可能需要查看 Empress - http://www.empress.com。在几个项目中使用过它，您可以配置加载多少。然而，它得到了 RDBMS 的开销，所以不确定它会像你想要的那样精简。
You might also consider MySQL with the Memcache addon. This allows you to use MySQL as a key value store. Much much faster than regular MySQL since you skip the SQL layer processing. Also, with MySQL, you can turn the knobs to play with how much memory is used.
您也可以考虑带有 Memcache 插件的 MySQL。这允许您使用 MySQL 作为键值存储。由于您跳过了 SQL 层处理，因此比常规 MySQL 快得多。此外，对于 MySQL，您可以转动旋钮来控制使用了多少内存。
Firebird is another low memory usage db - http://www.firebirdnews.org/docs/fb2min.html.
Firebird 是另一个低内存使用数据库 - http://www.firebirdnews.org/docs/fb2min.html。
Anyway, hope this helps. Without a more indepth explanation of your needs (is this embedded, why the need to save memory and if memory is precious what do you consider low memory consumption, do you need acid, redundancy, what do you consider lightning fast, etc.) its difficult to provide more of an analysis.
无论如何，希望这会有所帮助。没有对您的需求进行更深入的解释（这是嵌入式的，为什么需要节省内存，如果内存很宝贵，您认为内存消耗低，您是否需要酸，冗余，您认为闪电般的快速等）其很难提供更多的分析。
回答by kris.jeong
Why don't you use MySQL(or MariaDB) with Master-slave replication. Based on your requirements. MySql's master-slave architecture is fit for you.
为什么不将 MySQL（或 MariaDB）与主从复制一起使用。根据您的要求。MySql 的主从架构很适合你。
Basically, NoSQL need a lot of server. For example, MongoDB's minimal setting needs three server, HBase needs four server.
基本上，NoSQL 需要很多服务器。比如MongoDB的最小设置需要三台服务器，HBase需要四台服务器。
In this point of view, If you need more readability then add a new slave server on mysql architecture.
从这个角度来看，如果你需要更多的可读性，那么在 mysql 架构上添加一个新的从服务器。
We assume that mysql's read performance is 2k tps. Then four node of mysql's read performance is 8k tps.
我们假设 mysql 的读取性能是 2k tps。那么mysql的四个节点的读性能是8k tps。
It depends on your test result and service usage(read/write ratio).
这取决于您的测试结果和服务使用情况（读/写比率）。
check below link, that is "Marco Cecconi - The Architecture of StackOverflow". http://www.youtube.com/watch?v=t6kM2EM6so4
检查下面的链接，即“Marco Cecconi - StackOverflow 的架构”。 http://www.youtube.com/watch?v=t6kM2EM6so4
