Java 分布式并发控制
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/92452/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Distributed Concurrency Control
提问by Bob Gettys
I've been working on this for a few days now, and I've found several solutions but none of them incredibly simple or lightweight. The problem is basically this: We have a cluster of 10 machines, each of which is running the same software on a multithreaded ESB platform. I can deal with concurrency issues between threads on the same machine fairly easily, but what about concurrency on the same data on different machines?
我已经为此工作了几天,我找到了几个解决方案,但没有一个非常简单或轻量级。问题基本上是这样的:我们有一个由 10 台机器组成的集群,每台机器都在多线程 ESB 平台上运行相同的软件。我可以很容易地处理同一台机器上线程之间的并发问题,但是不同机器上相同数据的并发问题呢?
Essentially the software receives requests to feed a customer's data from one business to another via web services. However, the customer may or may not exist yet on the other system. If it does not, we create it via a web service method. So it requires a sort of test-and-set, but I need a semaphore of some sort to lock out the other machines from causing race conditions. I've had situations before where a remote customer was created twice for a single local customer, which isn't really desirable.
本质上,该软件接收请求,通过 Web 服务将客户的数据从一个企业传送到另一个企业。但是,该客户可能还存在,也可能不存在于另一个系统上。如果没有,我们通过网络服务方法创建它。所以它需要一种测试和设置,但我需要某种信号量来阻止其他机器导致竞争条件。我之前遇到过为一个本地客户创建两次远程客户的情况,这并不是真正可取的。
Solutions I've toyed with conceptually are:
我在概念上玩过的解决方案是:
Using our fault-tolerant shared file system to create "lock" files which will be checked for by each machine depending on the customer
Using a special table in our database, and locking the whole table in order to do a "test-and-set" for a lock record.
Using Terracotta, an open source server software which assists in scaling, but uses a hub-and-spoke model.
Using EHCache for synchronous replication of my in-memory "locks."
使用我们的容错共享文件系统创建“锁定”文件,每台机器将根据客户检查这些文件
在我们的数据库中使用一个特殊的表,并锁定整个表以便对锁定记录进行“测试和设置”。
使用 Terracotta,这是一种有助于扩展的开源服务器软件,但使用中心辐射模型。
使用 EHCache 同步复制我的内存“锁”。
I can't imagine that I'm the only person who's ever had this kind of problem. How did you solve it? Did you cook something up in-house or do you have a favorite 3rd-party product?
我无法想象我是唯一遇到过这种问题的人。你是怎么解决的?您是自己做饭还是有最喜欢的 3rd 方产品?
采纳答案by Boris Terzic
you might want to consider using Hazelcastdistributed locks. Super lite and easy.
您可能需要考虑使用Hazelcast分布式锁。超级精简和容易。
java.util.concurrent.locks.Lock lock = Hazelcast.getLock ("mymonitor");
lock.lock ();
try {
// do your stuff
}finally {
lock.unlock();
}
Hazelcast - Distributed Queue, Map, Set, List, Lock
Hazelcast - 分布式队列、映射、集合、列表、锁
回答by Clinton Pierce
Back in the day, we'd use a specific "lock server" on the network to handle this. Bleh.
过去,我们会使用网络上的特定“锁定服务器”来处理此问题。布莱。
Your database server might have resources specifically for doing this kind of thing. MS-SQL Server has application locks usable through the sp_getapplock/sp_releaseapplockprocedures.
您的数据库服务器可能有专门用于执行此类操作的资源。MS-SQL Server 具有可通过sp_getapplock/ sp_releaseapplock过程使用的应用程序锁。
回答by entzik
I made a simple RMI service with two methods: lock and release. both methods take a key (my data model used UUIDs as pk so that was also the locking key).
我用两种方法制作了一个简单的 RMI 服务:锁定和释放。这两种方法都需要一个密钥(我的数据模型使用 UUID 作为 pk,所以这也是锁定密钥)。
RMI is a good solution for this because it's centralized. you can't do this with EJBs (specialially in a cluster as you don't know on which machine your call will land). plus, it's easy.
RMI 是一个很好的解决方案,因为它是集中的。你不能用 EJB 来做到这一点(特别是在集群中,因为你不知道你的调用将登陆哪台机器)。另外,这很容易。
it worked for me.
它对我有用。
回答by Craig Day
I have done a lot of work with Coherence, which allowed several approaches to implementing a distributed lock. The naive approach was to request to lock the same logical object on all participating nodes. In Coherence terms this was locking a key on a Replicated Cache. This approach doesn't scale that well because the network traffic increases linearly as you add nodes. A smarter way was to use a Distributed Cache, where each node in the cluster is naturally responsible for a portion of the key space, so locking a key in such a cache always involved communication with at most one node. You could roll your own approach based on this idea, or better still, get Coherence. It really is the scalability toolkit of your dreams.
我在 Coherence 方面做了很多工作,它允许使用多种方法来实现分布式锁。天真的方法是请求在所有参与节点上锁定相同的逻辑对象。在 Coherence 术语中,这是锁定复制缓存上的键。这种方法不能很好地扩展,因为网络流量随着您添加节点而线性增加。一种更聪明的方法是使用分布式缓存,其中集群中的每个节点自然负责一部分键空间,因此在这样的缓存中锁定键总是涉及与最多一个节点的通信。您可以根据这个想法推出自己的方法,或者更好的是,获得 Coherence。它确实是您梦想中的可扩展性工具包。
I would add that any half decent multi-node network based locking mechanism would have to be reasonably sophisticated to act correctly in the event of any network failure.
我要补充的是,任何基于多节点网络的半体面的锁定机制都必须相当复杂才能在发生任何网络故障时正确操作。
回答by Boris Terzic
Not sure if I understand the entire context but it sounds like you have 1 single database backing this? Why not make use of the database's locking: if creating the customer is a single INSERT then this statement alone can serve as a lock since the database will reject a second INSERT that would violate one of your constraints (e.g. the fact that the customer name is unique for example).
不确定我是否理解整个上下文,但听起来您有 1 个单个数据库支持这个?为什么不利用数据库的锁定:如果创建客户是一个单一的 INSERT 那么这个语句就可以作为一个锁,因为数据库将拒绝第二个会违反你的约束之一的 INSERT(例如客户名称是例如独特)。
If the "inserting of a customer" operation is not atomic and is a batch of statements then I would introduce (or use) an initial INSERT that creates some simple basic record identifying your customer (with the necessary UNIQUEness constraints) and then do all the other inserts/updates in the same transaction. Again the database will take care of consistency and any concurrent modifications will result in one of them failing.
如果“插入客户”操作不是原子操作而是一组语句,那么我将引入(或使用)一个初始 INSERT,它创建一些简单的基本记录来标识您的客户(具有必要的 UNIQUEness 约束),然后执行所有同一事务中的其他插入/更新。同样,数据库将负责一致性,任何并发修改都将导致其中之一失败。
回答by Taylor Gautier
Terracotta is closer to a "tiered" model - all client applications talk to a Terracotta Server Array (and more importantly for scale they don't talk to one another). The Terracotta Server Array is capable of being clustered for both scale and availability (mirrored, for availability, and striped, for scale).
Terracotta 更接近于“分层”模型——所有客户端应用程序都与一个 Terracotta 服务器阵列通信(更重要的是,对于规模而言,它们不相互通信)。Terracotta 服务器阵列能够在规模和可用性方面进行集群(镜像,可用性和条带化,规模)。
In any case as you probably know Terracotta gives you the ability to express concurrency across the cluster the same way you do in a single JVM by using POJO synchronized/wait/notify or by using any of the java.util.concurrent primitives such as ReentrantReadWriteLock, CyclicBarrier, AtomicLong, FutureTask and so on.
在任何情况下,您可能知道 Terracotta 使您能够像在单个 JVM 中一样通过使用 POJO 同步/等待/通知或使用任何 java.util.concurrent 原语(例如 ReentrantReadWriteLock)来表达跨集群的并发性、CyclicBarrier、AtomicLong、FutureTask 等。
There are a lot of simple recipes demonstrating the use of these primitives in the Terracotta Cookbook.
在Terracotta Cookbook中有很多简单的食谱演示了这些原语的使用。
As an example, I will post the ReentrantReadWriteLock example (note there is no "Terracotta" version of the lock - you just use normal Java ReentrantReadWriteLock)
作为示例,我将发布 ReentrantReadWriteLock 示例(请注意,没有“Terracotta”版本的锁 - 您只需使用普通的 Java ReentrantReadWriteLock)
import java.util.concurrent.locks.*;
public class Main
{
public static final Main instance = new Main();
private int counter = 0;
private ReentrantReadWriteLock rwl = new ReentrantReadWriteLock(true);
public void read()
{
while (true) {
rwl.readLock().lock();
try {
System.out.println("Counter is " + counter);
} finally {
rwl.readLock().unlock();
}
try { Thread.currentThread().sleep(1000); } catch (InterruptedException ie) { }
}
}
public void write()
{
while (true) {
rwl.writeLock().lock();
try {
counter++;
System.out.println("Incrementing counter. Counter is " + counter);
} finally {
rwl.writeLock().unlock();
}
try { Thread.currentThread().sleep(3000); } catch (InterruptedException ie) { }
}
}
public static void main(String[] args)
{
if (args.length > 0) {
// args --> Writer
instance.write();
} else {
// no args --> Reader
instance.read();
}
}
}
回答by fern
We use Terracotta, so I would like to vote for that.
我们使用兵马俑,所以我想投票支持。
I've been following Hazelcast and it looks like another promising technology, but can't vote for it since I've not used it, and knowing that it uses a P2P based system at its heard, I really would not trust it for large scaling needs.
我一直在关注 Hazelcast,它看起来像是另一种很有前途的技术,但因为我没有使用过它,所以不能投票支持它,并且知道它使用的是基于 P2P 的系统,我真的不会相信它缩放需求。
But I have also heard of Zookeeper, which came out of Yahoo, and is moving under the Hadoop umbrella. If you're adventurous trying out some new technology this really has lots of promise since it's very lean and mean, focusing on just coordination. I like the vision and promise, though it might be too green still.
但我也听说过 Zookeeper,它来自 Yahoo,正在 Hadoop 保护伞下移动。如果您喜欢冒险尝试一些新技术,这确实有很大的希望,因为它非常精简和吝啬,只关注协调。我喜欢这个愿景和承诺,尽管它可能仍然过于绿色。
回答by Javier
I was going to advice on using memcached as a very fast, distributed RAM storage for keeping logs; but it seems that EHCache is a similar project but more java-centric.
我打算建议使用 memcached 作为一种非常快速的分布式 RAM 存储来保存日志;但似乎 EHCache 是一个类似的项目,但更以 Java 为中心。
Either one is the way to go, as long as you're sure to use atomic updates (memcached supports them, don't know about EHCache). It's by far the most scalable solution.
任何一种都是可行的,只要您确定使用原子更新(memcached 支持它们,不知道 EHCache)。这是迄今为止最具可扩展性的解决方案。
As a related datapoint, Google uses 'Chubby', a fast, RAM-based distributed lock storage as the root of several systems, among them BigTable.
作为相关数据点,Google 使用“Chubby”,一种快速的、基于 RAM 的分布式锁存储作为多个系统的根,其中包括 BigTable。
回答by Jonathan
If you can set up your load balancing so that requests for a single customer always get mapped to the same server then you can handle this via local synchronization. For example, take your customer ID mod 10 to find which of the 10 nodes to use.
如果您可以设置负载平衡,以便对单个客户的请求始终映射到同一台服务器,那么您可以通过本地同步来处理此问题。例如,使用您的客户 ID mod 10 来查找要使用的 10 个节点中的哪一个。
Even if you don't want to do this in the general case your nodes could proxy to each other for this specific type of request.
即使您不想在一般情况下这样做,您的节点也可以针对这种特定类型的请求相互代理。
Assuming your users are uniform enough (i.e. if you have a ton of them) that you don't expect hot spots to pop up where one node gets overloaded, this should still scale pretty well.
假设您的用户足够统一(即,如果您有大量用户),您不希望在一个节点过载的地方出现热点,这应该仍然可以很好地扩展。
回答by Kamran
We have been developing an open source, distributed synchronization framework, currently DistributedReentrantLock and DistributedReentrantReadWrite lock has been implemented, but still are in testing and refactoring phase. In our architecture lock keys are devided in buckets and each node is resonsible for certain number of buckets. So effectively for a successfull lock requests, there is only one network request. We are also using AbstractQueuedSynchronizer class as local lock state, so all the failed lock requests are handled locally, this drastically reduces network trafic. We are using JGroups (http://jgroups.org) for group communication and Hessian for serialization.
我们一直在开发一个开源的分布式同步框架,目前已经实现了 DistributedReentrantLock 和 DistributedReentrantReadWrite 锁,但还在测试和重构阶段。在我们的架构中,锁密钥被划分在桶中,每个节点负责一定数量的桶。因此,对于成功的锁定请求,有效地只有一个网络请求。我们还使用 AbstractQueuedSynchronizer 类作为本地锁状态,因此所有失败的锁请求都在本地处理,这大大减少了网络流量。我们使用 JGroups ( http://jgroups.org) 进行组通信,使用 Hessian 进行序列化。
for details, please check out http://code.google.com/p/vitrit/.
有关详细信息,请查看http://code.google.com/p/vitrit/。
Please send me your valuable feedback.
请将您的宝贵意见发送给我。
Kamran
卡姆兰