Java并发编程（1）—— 实现一个生产者消费者队列（三种方式）
生产者消费者队列，顾名思义，就是一个队列，不停地有生产者在里面生产对象并通知阻塞的消费者可以消费了，如果队列满了，生产者就阻塞不能再生产；消费者来消费（也就是读取并拿走队列里的对象）并通知阻塞的生产者，直到把队列消费空，就阻塞不能再消费。
2020.3.2更新：
原来的代码不是特别简练，修改了一下，参考的是youtube上一个博主的实现，实现的还是挺简练的。
BlockingQueue版本的实现
public class ProducerConsumerQueueBlockingQueue {
/** BlockingQueue实现的生产者消费者模式 **/
public static void main(String[] args) {
BlockingQueue<Object> blockingQueue = new ArrayBlockingQueue<>(10);
final Runnable producer = () -> {
while(true){
try {
//blockingQueue.put(ItemFactory.createItem());
blockingQueue.put(new Object());
}catch (InterruptedException e){
e.printStackTrace();
}
}
};
for (int i=0; i<5; ++i)
new Thread(producer).start();
final Runnable consumer = () -> {
while(true){
try {
Object item = blockingQueue.take();
// process(item);
}catch (InterruptedException e){
e.printStackTrace();
}
}
};
for (int i=0; i<5; ++i)
new Thread(consumer).start();
}
}
Condition的await/signal版本实现
public class ProducerConsumerQueue<E> {
private Queue<E> queue;
private int max = 16;
private Object notFull;
private Object notEmpty;
public ProducerConsumerQueue(int size){
queue = new LinkedList<>();
this.max = size;
notFull = new Object();
notEmpty = new Object();
}
public synchronized void put(E element){
while(queue.size() == max){
try {
notFull.wait();
}catch (InterruptedException e){
e.printStackTrace();
}
}
queue.add(element);
notEmpty.notifyAll();
}
public synchronized E take(){
while(queue.size()==0){
try {
notEmpty.wait();
}catch (InterruptedException e){
e.printStackTrace();
}
}
E element = queue.remove();
notFull.notifyAll();
return element;
}
public static void main(String[] args) {
ProducerConsumerQueue<Object> pc = new ProducerConsumerQueue(10);
final Runnable producer = ()->{
while(true)
pc.put(new Object());
};
final Runnable consumer = ()->{
while(true){
Object obj = pc.take();
// process(obj)
}
};
new Thread(producer).start();
new Thread(consumer).start();
}
}
wait/notify版本的实现
public class ProducerConsumerQueueCondition<E> {
private Queue<E> queue;
private int max = 16;
private ReentrantLock lock = new ReentrantLock();
private Condition notFull = lock.newCondition();
private Condition notEmpty = lock.newCondition();
public ProducerConsumerQueueCondition(int size){
queue = new LinkedList<>();
this.max = size;
}
public void put(E element){
lock.lock();
try{
while (queue.size() == max){
notFull.await();
}
queue.add(element);
notEmpty.signalAll();
}catch (InterruptedException e){
e.printStackTrace();
}finally {
lock.unlock();
}
}
public E take(){
E element = null;
lock.lock();
try{
while(queue.isEmpty()){
notEmpty.await();
}
element = queue.remove();
notFull.signalAll();
}catch (InterruptedException e){
e.printStackTrace();
}finally {
lock.unlock();
return element;
}
}
public static void main(String[] args) {
ProducerConsumerQueueCondition<Object> pc = new ProducerConsumerQueueCondition<>(10);
final Runnable producer = ()->{
while(true){
pc.put(new Object());
}
};
int producer_num = 3;
for (int i=0; i<producer_num; ++i)
new Thread(producer).start();
final Runnable consumer = ()->{
while(true){
Object item = pc.take();
// process(item)
}
};
int consumer_num = 3;
for (int i=0; i<consumer_num; ++i)
new Thread(consumer).start();
}
}
原来的实现：
wait/notify机制
wait/notify机制原理
wait/notify机制是与Monitor监视器锁关联在一起的。一个线程在持有某个对象的监视器锁之后，（在synchronized方法或代码块内），调用wait方法可以把锁释放（把Monitor对象的owner线程设置为null，计数器减一），让给其他堵塞在锁池的线程，自己则进入等待池，进入waiting状态。某个线程抢到对象锁之后，开始执行任务，执行到某一步的时候调用 notify() 或 notifyAll() 方法，notify方法会随机挑一个在等待池的线程，把他移出等待池，进入锁池，获得竞争锁的资格；如果是 notifyAll() 就是把所有等待池的线程全部移到锁池。接着当前线程不会释放锁，把任务执行完毕后，再释放锁，然后锁池的线程去竞争锁。
wait/notify 实现消费者生产者队列
package com.ds.Concurrent;
import java.util.LinkedList;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
public class ProducerConsumerQueue<T> {
// workload是一共要生产的产品数量，如果不设置workload这个类会无限运行下去
static AtomicInteger workLoad = new AtomicInteger(100);
public static void main(String[] args) {
LinkedList<Integer> list = new LinkedList<>();
int capacity = 8;
ExecutorService service = Executors.newFixedThreadPool(15);
for (int i=0; i<5; ++i){
service.submit(new Producer(list, capacity));
}
for(int i=0; i<5; ++i){
service.submit(new Consumer(list));
}
while (true){
if (workLoad.get()<=0 && list.isEmpty()){
service.shutdownNow();
return;
}
}
}
public static class Producer implements Runnable{
private final LinkedList<Integer> list;
private final int capacity;
public Producer(LinkedList<Integer> list, int capacity){
this.list = list;
this.capacity = capacity;
}
@Override
public void run(){
while(workLoad.get() > 0){
synchronized (list){
while (list.size() == capacity){
try {
list.wait();
}catch (InterruptedException e){
e.printStackTrace();
}
}
if (workLoad.get() > 0){
Integer num = new Random().nextInt();
list.add(num);
workLoad.decrementAndGet();
System.out.println(Thread.currentThread().getName() + " produce " +num + "workload:" + workLoad);
list.notifyAll();
}
}
}
}
}
public static class Consumer implements Runnable{
private final LinkedList<Integer> list;
public Consumer(LinkedList<Integer> list){
this.list = list;
}
@Override
public void run(){
while(true){
synchronized (list){
while (list.isEmpty()){
if (workLoad.get()<=0)
return;
try {
list.wait();
}catch (InterruptedException e){
e.printStackTrace();
}
}
int num = list.removeFirst();
System.out.println(Thread.currentThread().getName() + " consume " + num);
list.notifyAll();
if (workLoad.get()<=0 && list.isEmpty())
return;
}
}
}
}
}
最终输出的一部分如下：
要注意的地方是在判断阻塞条件的时候用的不是if而是while语句。因为若是用if语句，如果一个消费者线程消费完队列里最后一个数据后，下一个拿到锁的线程还是消费者线程，从wait()语句后面开始执行，就会再次在空队列里消费，而不是阻塞。所以需要while语句，哪怕wait之后又被notify，还是要重复判断阻塞条件。
Condition的await/signal机制
wait/notify机制是和监视器锁、synchronized关键字息息相关的；而Condition的await/signal机制则是和AQS（实现ReentrantLock、读写锁、CountdownLatch、CyclicBarrier等java并发工具的核心组件）、ReentrantLock相关联的。
Condition的await/signal机制与wait/notify机制相比，最大的特点是等待与通知的细粒度、定制化，因不同条件而进入等待状态的线程根据他们等待的原因，被分成组；在通知的时候，则是可以选择通知特定组的线程。
import java.util.LinkedList;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
public class ProducerConsumerQueueCondition {
private final static LinkedList<Integer> list = new LinkedList<>();
private final static ReentrantLock lock = new ReentrantLock();
private final static Condition full = lock.newCondition();
private final static Condition empty = lock.newCondition();
private final static int capacity = 10;
private volatile static int workload = 100;
public static void main(String[] args) {
ExecutorService service = Executors.newFixedThreadPool(10);
for (int i=0; i<5; ++i)
service.submit(new Producer());
for (int i=0; i<5; ++i)
service.submit(new Consumer());
while(true){
if (workload==0 && list.isEmpty()){
service.shutdownNow();
return;
}
}
}
private static class Producer implements Runnable{
@Override
public void run(){
while(workload > 0){
lock.lock();
try{
while(list.size() == capacity){
full.await();
}
if (workload > 0){
Integer num = new Random().nextInt();
list.add(num);
workload--;
System.out.println(Thread.currentThread().getName() + " produce " + num + "workload: " + workload);
empty.signal();
}
}catch (InterruptedException e){
e.printStackTrace();
}finally {
lock.unlock();
}
}
}
}
private static class Consumer implements Runnable{
@Override
public void run(){
while(list.size()>0 || workload>0){
try{
lock.lock();
while (list.size() == 0){
if (workload == 0)
return;
empty.await();
}
int num = list.removeFirst();
System.out.println(Thread.currentThread().getName() + " consume " + num);
full.signal();
}catch (InterruptedException e){
e.printStackTrace();
}finally {
lock.unlock();
}
}
}
}
}
最终输入结果一部分如下：
BlockingQueue实现
前两种实现方式都是向线程池提交线程，如果主线程不监听workload和list长度，手工停止线程池，程序会一直运行不结束。
在下面的实现中，直接开辟Thread对象启动，在run方法里调出while循环的时候return（我感觉这个return并没有实际作用，但是如果不加上的话，等了一会线程还是没有结束，这个有点奇怪，没搞懂为什么），就不需要自己去终止线程池并且返回。
下面的实现里依旧加入了workload，如果只是BlockingQueue实现生产者消费者模式的核心代码的话，可以更精简，删除workload。
package com.ds.Concurrent;
import java.util.Random;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class ProducerConsumerQueueBlockingQueue {
/** BlockingQueue实现的生产者消费者模式 **/
private static final LinkedBlockingQueue<Integer> blockinQueue = new LinkedBlockingQueue<>(20);
private static AtomicInteger workload = new AtomicInteger(100);
public static void main(String[] args) {
for (int i=0; i<5; ++i)
new Thread(new Producer()).start();
for (int i=0; i<5; ++i)
new Thread(new Consumer()).start();
}
private static class Producer implements Runnable{
@Override
public void run() {
while (workload.get() > 0){
try{
if(workload.get()>0){
blockinQueue.put(new Random().nextInt());
workload.decrementAndGet();
System.out.println("produce, workload: " + workload);
}
}catch (InterruptedException e){
e.printStackTrace();
}
}
return;
}
}
private static class Consumer implements Runnable{
@Override
public void run(){
while(!blockinQueue.isEmpty() || workload.get()>0){
try{
int num = blockinQueue.take();
System.out.println("consume " + num);
}catch (InterruptedException e){
e.printStackTrace();
}
}
return;
}
}
}
前两种方法产生的输出，workload都是从99到0依次减少的，但是BlockingQueue版本的输出是这样的：
这里只取了最后一段的一部分，可以看到workload的减少输出是乱序的，这一点也比较奇怪，要再去瞅瞅BlockingQueue的源码了。