Python queue源码解析，如何写线程安全的队列

python 的队列实现方式很简单，通过互斥锁+条件变量的方式实现，这种方式实在是太经典了。最近看的《现代操作系统》中，在线程通信的互斥量一节就有提及互斥量+条件变量来解决生产者消费者问题。（参见《现代操作系统》第4版P76-P77）

互斥量，不必多说，如python中的threading.lock()，也就是我们最常用的锁。它限制了同一个时刻，只能有一个线程进入临界区中，保证了临界区的线程安全。

条件变量，它满足了当线程满足一定的条件才能得到锁。这说起来拗口，但如果我们往深的方向想，有了互斥量为什么还需要条件变量？举个例子，在生产者消费者模式中，如果我们的空槽数量只有1个，而我们的生产者和消费者都在同时竞争获得互斥量，而生产者已经获得了互斥量，但却发现了没有空槽，这时候只能释放互斥量，（否则消费者不能获得互斥量，就造成了死锁），但是你获得了互斥量，再检查了没有空槽，又释放了互斥量，这不是做了无用功吗？所以，得到的最好解决方法就是万一生产者发现没有空槽可用了，就将自己阻塞起来，等其他人在有空槽的时候唤醒自己，而这时候，就是条件变量起作用了。

条件变量经常和互斥量一起使用，这种模式用于让一个线程锁住一个互斥量，当它不能获得一个期待的结果，（例如生产者发现没有空槽），条件变量就能原子性地阻塞当前线程并解锁它持有地互斥量。所以，条件变量依赖于互斥量。

来看看源代码：（已删简，中文注释即为我加

class Queue:
    '''Create a queue object with a given maximum size.

    If maxsize is <= 0, the queue size is infinite.
    '''

    def __init__(self, maxsize=0):
        self.maxsize = maxsize
        self._init(maxsize)

        # mutex must be held whenever the queue is mutating.  All methods
        # that acquire mutex must release it before returning.  mutex
        # is shared between the three conditions, so acquiring and
        # releasing the conditions also acquires and releases mutex.
        self.mutex = threading.Lock()            #互斥量

        # Notify not_empty whenever an item is added to the queue; a
        # thread waiting to get is notified then.
        self.not_empty = threading.Condition(self.mutex)       #非空条件变量

        # Notify not_full whenever an item is removed from the queue;
        # a thread waiting to put is notified then.
        self.not_full = threading.Condition(self.mutex)        #非满条件变量，可以看出这两个条件变量都是依赖于同一个互斥量self.mutex

        # Notify all_tasks_done whenever the number of unfinished tasks
        # drops to zero; thread waiting to join() is notified to resume
        self.all_tasks_done = threading.Condition(self.mutex)
        self.unfinished_tasks = 0

    def put(self, item, block=True, timeout=None):
        '''Put an item into the queue.

        If optional args 'block' is true and 'timeout' is None (the default),
        block if necessary until a free slot is available. If 'timeout' is
        a non-negative number, it blocks at most 'timeout' seconds and raises
        the Full exception if no free slot was available within that time.
        Otherwise ('block' is false), put an item on the queue if a free slot
        is immediately available, else raise the Full exception ('timeout'
        is ignored in that case).
        '''
        with self.not_full:       #相当于with self.mutex
            if self.maxsize > 0:    #当前队列不设置最长长度
                if not block:    #当前队列是非阻塞型的话，抛出队列满异常
                    if self._qsize() >= self.maxsize:
                        raise Full
                elif timeout is None:    #当前队列不设置
                    while self._qsize() >= self.maxsize:   
                        self.not_full.wait()        #队列满，调用条件变量阻塞当前线程
                elif timeout < 0:
                    raise ValueError("'timeout' must be a non-negative number")
                else:
                    endtime = time() + timeout
                    while self._qsize() >= self.maxsize:
                        remaining = endtime - time()
                        if remaining <= 0.0:
                            raise Full
                        self.not_full.wait(remaining)  #等待remaining时间
            self._put(item)
            self.unfinished_tasks += 1
            self.not_empty.notify()                 #通知非空条件变量

    def get(self, block=True, timeout=None):
        '''Remove and return an item from the queue.

        If optional args 'block' is true and 'timeout' is None (the default),
        block if necessary until an item is available. If 'timeout' is
        a non-negative number, it blocks at most 'timeout' seconds and raises
        the Empty exception if no item was available within that time.
        Otherwise ('block' is false), return an item if one is immediately
        available, else raise the Empty exception ('timeout' is ignored
        in that case).
        '''
        with self.not_empty:
            if not block:
                if not self._qsize():
                    raise Empty
            elif timeout is None:
                while not self._qsize():
                    self.not_empty.wait()
            elif timeout < 0:
                raise ValueError("'timeout' must be a non-negative number")
            else:
                endtime = time() + timeout
                while not self._qsize():
                    remaining = endtime - time()
                    if remaining <= 0.0:
                        raise Empty
                    self.not_empty.wait(remaining)
            item = self._get()
            self.not_full.notify()
            return item

由于get与put是相对的，就不多写了。另附《现代操作系统》中对条件变量的运用：