一、背景
在分布式环境中,多线程中的各种锁是不生效的(由于程序在多台机器上运行),即使是单线程,一旦访问临界资源,也会出错的。
因此我们需要分布式锁来对临界资源进行同步访问。
在此,介绍基于zookeeper,curator实现的分布式锁,包括InterProcessMutex锁、InterProcessReadWriteLock读写锁、信号量InterProcessSemaphoreV2、
分布式栅栏DistributedBarrier。
二、添加maven依赖
org.apache.zookeeper zookeeper 3.4.6 org.apache.curator curator-framework 4.0.0 org.apache.curator curator-client 4.0.0 org.apache.curator curator-recipes 4.0.0
三、InterProcessMutex锁
通过InterProcessMutex创建锁,然后使用acquire()获取锁,使用release()释放锁,
代码如下:
import org.apache.curator.framework.CuratorFramework; import org.apache.curator.framework.CuratorFrameworkFactory; import org.apache.curator.framework.recipes.locks.InterProcessMutex; import org.apache.curator.retry.ExponentialBackoffRetry; import java.util.ArrayList; import java.util.List; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.function.Consumer; public class ZookeeperLockMain { public static void main(String[] args) throws Exception { //创建zookeeper客户端 CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3)); client.start(); //指定锁路径 String lockPath = "/zkLockRoot/lock_1"; //创建锁,为可重入锁,即是获锁后,还可以再次获取,本例以此为例 InterProcessMutex lock = new InterProcessMutex(client, lockPath); // //创建锁,为不可重入锁,即是获锁后,不可以再次获取,这里不作例子,使用和重入锁类似 // InterProcessSemaphoreMutex lock = new InterProcessSemaphoreMutex(client, lockPath); ExecutorService executor = Executors.newCachedThreadPool(); Consumer输出:consumer = (InterProcessMutex typeLock)->{ try{ List > callList = new ArrayList<>(); Callable call = () -> { try{ //获取锁 typeLock.acquire(); System.out.println(Thread.currentThread() + " acquire read lock"); }catch (Exception e){ }finally { //释放锁 typeLock.release(); System.out.println(Thread.currentThread() + " release read lock"); } return "true"; }; //5个并发线程 for (int i = 0; i < 5; i++) { callList.add(call); } List > futures = executor.invokeAll(callList); }catch (Exception e){ } }; //分布式锁测试 consumer.accept(lock); executor.shutdown(); } }
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] release read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] release read lock
结果分析:
可以发现,只有等前一个获取锁的线程释放锁后,下一个线程才能获取锁。
四、InterProcessReadWriteLock读写锁
首先通过InterProcessReadWriteLock创建读写锁,然后再通过readLock()获取读锁,通过writeLock()获取写锁。
最后基于读锁或写锁,使用acquire()获取锁,使用release()释放锁。
可以同时有多个线程获取读锁,但同时只能有一个线程获取写锁。
代码如下:
import org.apache.curator.framework.CuratorFramework; import org.apache.curator.framework.CuratorFrameworkFactory; import org.apache.curator.framework.recipes.locks.InterProcessMutex; import org.apache.curator.framework.recipes.locks.InterProcessReadWriteLock; import org.apache.curator.retry.ExponentialBackoffRetry; import java.util.ArrayList; import java.util.List; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.function.Consumer; public class ZookeeperReadWriteLockMain { public static void main(String[] args) throws Exception { //创建zookeeper客户端 CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3)); client.start(); //指定锁路径 String lockPath = "/zkLockRoot/lock_1"; //创建读写锁 InterProcessReadWriteLock lock = new InterProcessReadWriteLock(client, lockPath); //生成线程池 ExecutorService executor = Executors.newCachedThreadPool(); Consumer输出:consumer = (InterProcessMutex typeLock)->{ try{ List > callList = new ArrayList<>(); Callable call = () -> { try{ typeLock.acquire(); System.out.println(Thread.currentThread() + " acquire read lock"); }catch (Exception e){ }finally { typeLock.release(); System.out.println(Thread.currentThread() + " release read lock"); } return "true"; }; //5个并发线程 for (int i = 0; i < 5; i++) { callList.add(call); } List > futures = executor.invokeAll(callList); }catch (Exception e){ } }; //读锁测试(多个线程可同时获取读锁) System.out.println("5个并发线程,读锁测试"); InterProcessMutex readLock = lock.readLock(); consumer.accept(readLock); //写锁测试(同时只有一个线程获取写锁) System.out.println("5个并发线程,写锁测试"); InterProcessMutex writeLock = lock.writeLock(); consumer.accept(writeLock); executor.shutdown(); } }
5个并发线程,读锁测试
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-4,5,main] release read lock
Thread[pool-3-thread-2,5,main] release read lock
5个并发线程,写锁测试
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] release read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] release read lock
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
结果分析:
可以发现,同时多个线程可获取读锁,但同时只能有一个线程获取写锁。
五、InterProcessSemaphoreV2信息量
InterProcessSemaphoreV2信号量与多线程中的Semaphonre信息量含义是一致的,
即同时最多只能允许指定数量的线程访问临界资源。
通过InterProcessSemaphoreV2创建信息量,然后使用acquire()获取访问权限,
使用returnLease(lease)翻译访问权限。
代码如下:
import org.apache.curator.framework.CuratorFramework; import org.apache.curator.framework.CuratorFrameworkFactory; import org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2; import org.apache.curator.framework.recipes.locks.Lease; import org.apache.curator.retry.ExponentialBackoffRetry; import java.util.ArrayList; import java.util.List; import java.util.concurrent.*; import java.util.function.Consumer; public class ZookeeperSemaphoreMain { public static void main(String[] args) throws Exception { //创建zookeeper客户端 CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3)); client.start(); //指定锁路径 String lockPath = "/zkLockRoot/lock_1"; //创建信号量,指定同时最大访问数为3个 InterProcessSemaphoreV2 semaphoreLock = new InterProcessSemaphoreV2(client, lockPath, 3); //生成线程池 ExecutorService executor = Executors.newCachedThreadPool(); Consumerconsumer = (InterProcessSemaphoreV2 semaphore)->{ try{ List 5; i++) { callList.add(call); } List> callList = new ArrayList<>(); Callable call = () -> { Lease lease = semaphore.acquire(); try{ System.out.println(Thread.currentThread() + " acquire semaphore"); TimeUnit.MILLISECONDS.sleep(200); }catch (Exception e){ }finally { semaphore.returnLease(lease); System.out.println(Thread.currentThread() + " release semaphore"); } return "true"; }; //5个并发线程 for (int i = 0; i < > futures = executor.invokeAll(callList); }catch (Exception e){ } }; //读锁测试(多个线程可同时获取读锁) System.out.println("5个并发线程,信号量测试"); consumer.accept(semaphoreLock); executor.shutdown(); } }
输出:
5个并发线程,信号量测试
Thread[pool-3-thread-1,5,main] acquire semaphore
Thread[pool-3-thread-2,5,main] acquire semaphore
Thread[pool-3-thread-1,5,main] release semaphore
Thread[pool-3-thread-5,5,main] acquire semaphore
Thread[pool-3-thread-4,5,main] acquire semaphore
Thread[pool-3-thread-2,5,main] release semaphore
Thread[pool-3-thread-3,5,main] acquire semaphore
Thread[pool-3-thread-5,5,main] release semaphore
Thread[pool-3-thread-4,5,main] release semaphore
Thread[pool-3-thread-3,5,main] release semaphore
六、DistributedBarrier分布式栅栏
分布式栅栏DistributedBarrier用于在分布式环境下,阻塞指定数量线程(不一定在同一台机器),当这些线程达到某一点时,
再放开阻塞。和多线程编程中的CyclicBarrier类似。
barrier.waitOnBarrier()方法用于阻塞,当所有线程都调用了该方法后,阻塞解除。
示例代码:
import org.apache.curator.framework.CuratorFramework; import org.apache.curator.framework.CuratorFrameworkFactory; import org.apache.curator.framework.recipes.barriers.DistributedBarrier; import org.apache.curator.retry.ExponentialBackoffRetry; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.function.Consumer; public class ZookeeperDistributedBarrierMain { public static void main(String[] args) throws Exception { //创建zookeeper客户端 CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3)); client.start(); //指定锁路径 String lockPath = "/zkLockRoot/lock_1"; //创建分布式栅栏 DistributedBarrier distributedBarrier = new DistributedBarrier(client, lockPath); distributedBarrier.setBarrier(); //生成线程池 ExecutorService executor = Executors.newCachedThreadPool(); Consumer输出:consumer = (DistributedBarrier barrier)->{ try{ Callable call = () -> { try{ System.out.println(Thread.currentThread() + " rearch barrier, waiting"); barrier.waitOnBarrier(); System.out.println(Thread.currentThread() + " do next"); }catch (Exception e){ } return true; }; //5个并发线程 for (int i = 0; i < 5; i++) { executor.submit(call); } }catch (Exception e){ } }; //栅栏测试(多个线程栅栏测试) System.out.println("5个并发线程,栅栏测试"); consumer.accept(distributedBarrier); distributedBarrier.removeBarrier(); executor.shutdown(); client.close(); } }
5个并发线程,栅栏测试
Thread[pool-3-thread-2,5,main] rearch barrier, waiting
Thread[pool-3-thread-1,5,main] rearch barrier, waiting
Thread[pool-3-thread-4,5,main] rearch barrier, waiting
Thread[pool-3-thread-3,5,main] rearch barrier, waiting
Thread[pool-3-thread-5,5,main] rearch barrier, waiting
Thread[pool-3-thread-5,5,main] do next
Thread[pool-3-thread-3,5,main] do next
Thread[pool-3-thread-4,5,main] do next
Thread[pool-3-thread-1,5,main] do next
Thread[pool-3-thread-2,5,main] do next