“生产者/消费者”问题描述:
有一个有限缓冲区和两个线程:生产者和消费者。他们分别把产品放入缓冲区和从缓冲区中拿走产品。当一个生产者在缓冲区满时必须等待,当一个消费者在缓冲区空时也必须等待。
1. 单锁模型
#include"stdio.h"
运行结果
buffer is empty...consumer is waiting...
buffer is empty...consumer is waiting...
buffer is empty...consumer is waiting...
pruducter set the 0
pruducter set the 1
pruducter set the 2
pruducter set the 3
pruducter set the 4
pruducter set the 5
pruducter set the 6
pruducter set the 7
pruducter set the 8
pruducter set the 9
consumer get the 0
consumer get the 1
consumer get the 2
consumer get the 3
consumer get the 4
consumer get the 5
consumer get the 6
consumer get the 7
consumer get the 8
consumer get the 9
容易出现极端状况,一开始一直是consumer加锁,producer无法提供;不久后producer终于得到加锁,存入了10个货物,而后consumer取得10个货物。
2. 两个互斥锁
#include"stdio.h"
#include"pthread.h" int buffer[10]; int top = 0; int itime = 0; int itime2 = 0; pthread_t thread[2]; pthread_mutex_t mut; pthread_mutex_t mut2; void producer() { while(1) { if(itime == 10) return; pthread_mutex_lock(&mut); if(top == 10) { printf("buffer is full...producer is waiting...\n"); pthread_mutex_unlock(&mut2); continue; } printf("pruducter set the %d\n", itime); top++; itime++; pthread_mutex_unlock(&mut2); } } void consumer() { while(1) { if(itime2 == 10) return; pthread_mutex_lock(&mut2); if(top == 0) { printf("buffer is empty...consumer is waiting...\n"); pthread_mutex_unlock(&mut); continue; } printf("consumer get the %d\n", itime2); top--; itime2++; pthread_mutex_unlock(&mut); } } int main() { pthread_create(&thread[0], NULL, (void*)(&producer), NULL); pthread_create(&thread[1], NULL, (void*)(&consumer), NULL); sleep(1); return 0; }
buffer is empty...consumer is waiting...
pruducter set the 0
consumer get the 0
pruducter set the 1
consumer get the 1
pruducter set the 2
consumer get the 2
pruducter set the 3
consumer get the 3
pruducter set the 4
consumer get the 4
pruducter set the 5
consumer get the 5
pruducter set the 6
consumer get the 6
pruducter set the 7
consumer get the 7
pruducter set the 8
consumer get the 8
pruducter set the 9
consumer get the 9
很完美对不对?生产者生产一个就把自己锁起来,消费者消费一个后解锁生产者,把自己锁起来,生产者继续生产,循环反复。互斥锁的确能很好的实现进程/线程之间的同步问题,但是它是通过锁机制来实现的,就是仅仅通过加锁和解锁实现同步,效率比较低。
3. 利用条件变量
#include"stdio.h"
#include"pthread.h" int buffer[10]; int top = 0; int itime = 0; int itime2 = 0; pthread_t thread[2]; pthread_mutex_t mut; pthread_cond_t con, con2; void producer() { while(1) { if(itime == 10) return; pthread_mutex_lock(&mut); if(top == 10) { printf("buffer is full...producer is waiting...\n"); pthread_cond_wait(&con, &mut); } printf("pruducter set the %d\n", itime); top++; itime++; pthread_cond_signal(&con2); pthread_mutex_unlock(&mut); sleep(1); } } void consumer() { while(1) { if(itime2 == 10) return; pthread_mutex_lock(&mut); if(top == 0) { printf("buffer is empty...consumer is waiting...\n"); pthread_cond_wait(&con2, &mut); } printf("consumer get the %d\n", itime2); top--; itime2++; pthread_cond_signal(&con); pthread_mutex_unlock(&mut); sleep(1); } } int main() { pthread_create(&thread[0], NULL, (void*)(&producer), NULL); pthread_create(&thread[1], NULL, (void*)(&consumer), NULL); sleep(10); return 0; }
buffer is empty...consumer is waiting...
pruducter set the 0
consumer get the 0
buffer is empty...consumer is waiting...
pruducter set the 1
consumer get the 1
buffer is empty...consumer is waiting...
pruducter set the 2
consumer get the 2
buffer is empty...consumer is waiting...
pruducter set the 3
consumer get the 3
pruducter set the 4
consumer get the 4
pruducter set the 5
consumer get the 5
pruducter set the 6
consumer get the 6
pruducter set the 7
consumer get the 7
pruducter set the 8
consumer get the 8
pruducter set the 9
consumer get the 9
结果还算比较正常,理解一下变量的使用。消费者发现缓冲区没有东西,通过条件变量把自己锁住;生产者生产并激活消费者;消费者从缓冲区消费;当生产者发现缓冲区满的时候,通过条件变量把自己锁住,消费者消费并重新激活生产者。如此。
信号量实现
学习了信号量以及共享内存后,我们就可以实现进程的同步与互斥了。说到这里,最经典的例子莫过于生产者和消费者模型。下面就和大家一起分析,如何一步步实现这个经典模型。
下面程序,实现的是多个生产者和多个消费者对N个缓冲区(N个货架)进行访问的例子。现在先想想我们以前的伪代码是怎么写的?是不是这样:
//生产者:
while(1)
{
p(semid,1);
sleep(3);
p(semid,0);
//producer is producing a product
goods=rand()%10;
shmaddr[indexaddr[0]]=goods;
printf("producer:%d produces a product[%d]:%d\n",getpid(),indexaddr[0],goods);
indexaddr[0]=(indexaddr[0]+1)%10;
v(semid,0);
sleep(3);
v(semid,2);
}
//消费者:
while(1)
{
p(semid,2);
sleep(1);
p(semid,0);
//consumer is consuming a product
goods=shmaddr[indexaddr[1]];
printf("consumer:%d consumes a product[%d]:%d\n",getpid(),indexaddr[1],goods);
indexaddr[1]=(indexaddr[1]+1)%num;
v(semid,0);
sleep(1);
v(semid,1);
}
可能上面的代码你有些眼熟,又有些困惑,因为它和课本上的代码不完全一样,其实上面的代码就是伪代码的linuxC语言具体实现。我们从上面的代码中慢慢寻找伪代码的踪迹:p(semid,0)和v(semid,0)的作用是让进程互斥访问临界区。临界区中包含的数据indexaddr[0],indexaddr[1],以及shmaddr数组分别对应伪代码中的in,out,buffer。p(semid,1)和v(semid,2)以及p(semid,2)和v(semid,1)实现的是同步作用。
并且,在生产者中,生产者生产了一个货物(goods=rand()%10;),然后将这个货物放上货架(shmaddr[indexaddr[0]]=goods;)。在消费者中,消费和从货架上取下货物(goods=shmaddr[indexaddr[1]];)。
好了,现在再看一边上面的代码,我想你的思路就清晰了。
了解了核心代码,并不能算就完成了生产者和消费者模型,因为生产者和消费者核心代码前还得做一些些准备工作,具体要准备些什么,我们具体来分析。
首先申请一块共享内存,这块共享内存用于存放生产者所生产的货物。同时我们可以看到这块共享内存大小为10字节。这里需要注意,每个生产着或消费者运行后,都要去试着分配这样的一块共享内存。如果在当前进程运行前已经有某个进程已经创建了这块共享内存,那么这个进程就不再创建(此时createshm会返回-1并且错误代码为EEXIST),只是打开这块共享内存。创建后,再将这块共享内存添加到当前进程的地址空间。
num=10;
//create a shared memory as goods buffer
if((shmid_goods=createshm(".",'s',num))==-1)
{
if(errno==EEXIST)
{
if((shmid_goods=openshm(".",'s'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
//attach the shared memory to the current process
if((shmaddr=shmat(shmid_goods,(char*)0,0))==(char*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
接下来还要再申请一块共享内存,用于存放两个整形变量in和out(其实就是申请一个含有2个整形变量的数组而已)。他们记录的是生产和消费货物时“货架”的索引。与上面情况相同,如果已经有其他进程创建了此块共享内存,那么当前进程只是打开它而已。
注意这里对两个整形变量的初始化时的值均为0。
//create a shared memory as index
if((shmid_index=createshm(".",'z',2))==-1)
{
if(errno==EEXIST)
{
if((shmid_index=openshm(".",'z'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
else
{
is_noexist=1;
}
//attach the shared memory to the current process
if((indexaddr=shmat(shmid_index,(int*)0,0))==(int*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
if(is_noexist)
{
indexaddr[0]=0;
indexaddr[1]=0;
}
接下来就是创建一个信号量集,这个信号量集中包含三个信号量。第一个信号量实现的互斥作用,即进程对临界区的互斥访问。剩下两个均实现的是同步作用,协调生产者和消费者的合理运行,即货架上没有空位时候生产者不再生产,货架上无商品时消费者不再消费。
注意下面对每个信号量的赋值情况。互斥信号量当然初值为1。而同步信号量两者之和不能大于num的值。
//create a semaphore set including 3 semaphores
if((semid=createsem(".",'t',3,0))==-1)
{
if(errno==EEXIST)
{
if((semid=opensem(".",'t'))==-1)
{
exit(1);
}
}
else
{
perror("semget error:");
exit(1);
}
}
else
{
union semun arg;
//seting value for mutex semaphore
arg.val=1;
if(semctl(semid,0,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//set value for synchronous semaphore
arg.val=num;
//the num means that the producer can continue to produce num products
if(semctl(semid,1,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//the last semaphore's value is default
//the default value '0' means that the consumer is not use any product now
}
基本上这样,就算完成了生产者和消费者的前期工作。我们可以看到,在核心代码中,我们只需要“装模作样”的将代码“各就各位”即可,当然这需要你理解生产者消费者这个基本模型。而在下面的准备代码中,则需要我们理解关于信号量和共享内存的一些基本函数。
最后再说说使用,建议先运行一个生产者和一个消费者,观察两者是如何协调工作的。然后再只运行一个生产者或一个消费者,看其是否会阻塞。了解了以上情况后,你就可以同时运行多个生产者和消费者了。
下面是源代码:
shm.h
#include
producer.c
#include "shm.h"
int main()
{
int num;
int shmid_goods,shmid_index,semid;
char* shmaddr=NULL;
int *indexaddr=NULL;
int is_noexist=0;
num=10;
//create a shared memory as goods buffer
if((shmid_goods=createshm(".",'s',num))==-1)
{
if(errno==EEXIST)
{
if((shmid_goods=openshm(".",'s'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
//attach the shared memory to the current process
if((shmaddr=shmat(shmid_goods,(char*)0,0))==(char*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
//create a shared memory as index
if((shmid_index=createshm(".",'z',2))==-1)
{
if(errno==EEXIST)
{
if((shmid_index=openshm(".",'z'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
else
{
is_noexist=1;
}
//attach the shared memory to the current process
if((indexaddr=shmat(shmid_index,(int*)0,0))==(int*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
if(is_noexist)
{
indexaddr[0]=0;
indexaddr[1]=0;
}
//create a semaphore set including 3 semaphores
if((semid=createsem(".",'t',3,0))==-1)
{
if(errno==EEXIST)
{
if((semid=opensem(".",'t'))==-1)
{
exit(1);
}
}
else
{
perror("semget error:");
exit(1);
}
}
else
{
union semun arg;
//seting value for mutex semaphore
arg.val=1;
if(semctl(semid,0,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//set value for synchronous semaphore
arg.val=num;
//the num means that the producer can continue to produce num products
if(semctl(semid,1,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//the last semaphore's value is default
//the default value '0' means that the consumer is not use any product now
}
int goods=0;
while(1)
{
p(semid,1);
sleep(3);
p(semid,0);
//producer is producing a product
goods=rand()%10;
shmaddr[indexaddr[0]]=goods;
printf("producer:%d produces a product[%d]:%d\n",getpid(),indexaddr[0],goods);
indexaddr[0]=(indexaddr[0]+1)%10;
v(semid,0);
sleep(3);
v(semid,2);
}
}
consumer.c
#include "shm.h"
int main(int argc,char **argv)
{
int num;
int shmid_goods,shmid_index,semid;
char* shmaddr=NULL;
int* indexaddr=NULL;
int is_noexist=0;
num=10;
//create a shared memory as goods buffer
if((shmid_goods=createshm(".",'s',num))==-1)
{
if(errno==EEXIST)
{
if((shmid_goods=openshm(".",'s'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
//attach the shared memory to the current process
if((shmaddr=shmat(shmid_goods,(char*)0,0))==(char*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
//create a shared memory as index
if((shmid_index=createshm(".",'z',2))==-1)
{
if(errno==EEXIST)
{
if((shmid_index=openshm(".",'z'))==-1)
{
exit(1);
}
}
else
{
perror("create shared memory failed\n");
exit(1);
}
}
else
{
is_noexist=1;
}
//attach the shared memory to the current process
if((indexaddr=shmat(shmid_index,(int*)0,0))==(int*)-1)
{
perror("attach shared memory error\n");
exit(1);
}
if(is_noexist)
{
indexaddr[0]=0;
indexaddr[1]=0;
}
//create a semaphore set including 3 semaphores
if((semid=createsem(".",'t',3,0))==-1)
{
if(errno==EEXIST)
{
if((semid=opensem(".",'t'))==-1)
{
exit(1);
}
}
else
{
perror("semget error:");
exit(1);
}
}
else
{
union semun arg;
//seting value for mutex semaphore
arg.val=1;
if(semctl(semid,0,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//set value for synchronous semaphore
arg.val=num;
//the num means that the producer can continue to produce num products
if(semctl(semid,1,SETVAL,arg)==-1)
{
perror("setting semaphore value failed\n");
return -1;
}
//the last semaphore's value is default
//the default value '0' means that the consumer is not use any product now
}
int goods=0;
while(1)
{
p(semid,2);
sleep(1);
p(semid,0);
//consumer is consuming a product
goods=shmaddr[indexaddr[1]];
printf("consumer:%d consumes a product[%d]:%d\n",getpid(),indexaddr[1],goods);
indexaddr[1]=(indexaddr[1]+1)%num;
v(semid,0);
sleep(1);
v(semid,1);
}
}
生产者消费者问题是操作系统中的一个经典的问题。
他描述的是一个,多个生产者与多个消费者共享多个缓冲区的事情,具体的定义百度。
然后看了操作系统的书籍如何解决书上给的伪代码是这样的
item B[k];
semaphore empty; empty=k; //可以使用的空缓冲区数
semaphore full; full=0; //缓冲区内可以使用的产品数
semaphore mutex; mutex=1; //互斥信号量
int in=0; //放入缓冲区指针
int out=0; //取出缓冲区指针
cobegin
process producer_i ( ) { process consumer_j( ) {
while(true) { while(true) {
produce( ); P(full);
P(empty); P(mutex);
P(mutex); take( ) from B[out];
append to B[in]; V(empty);
in=(in+1)%k; out=(out+1)%k;
V(mutex); V(mutex);
V(full); consume( );
} }
} }
coend
上面的注释,和过程已经比较到位了,只是我习惯用我的方法,即把生产和消费,放入临界区所以下面是我解决生产消费模型所用的伪代码
item B[k];
semaphore empty; empty=k; //可以使用的空缓冲区数
semaphore full; full=0; //缓冲区内可以使用的产品数
semaphore mutex; mutex=1; //互斥信号量
int in=0; //放入缓冲区指针
int out=0; //取出缓冲区指针
cobegin
process producer_i ( ) { process consumer_j( ) {
while(true) { while(true) {
P(empty); P(full);
P(mutex); P(mutex);
produce( ); take( ) from B[out];
append to B[in]; consume( );
in=(in+1)%k; out=(out+1)%k;
V(mutex); V(mutex);
V(full); V(empty);
} }
} }
coend
好了说了这么多我该帖下我的代码了,此代码在Linux环境下的多线程操作,用到了信号量的。。。
#include
#include
#include
#include
#include
#include
#include
#include
#define ERR_EXIT(m) \
do \
{ \
perror(m); \
exit(EXIT_FAILURE); \
} while(0)
#define CONSUMERS_COUNT 2 //消费者人数
#define PRODUCERS_COUNT 2 //生产者人数
#define BUFFSIZE 5
int g_buffer[BUFFSIZE]; //缓冲区数目
unsigned short in = 0; //放入产品的指针(生产到哪个缓冲区)
unsigned short out = 0; //取出缓冲区指针 (在哪个缓冲区消费的)
unsigned short produce_id = 0;
unsigned short consume_id = 0;
sem_t g_sem_full; //可以使用的空缓冲区数(缓冲区中可以生产多少产品)
sem_t g_sem_empty; //缓冲区内可以使用的产品数(可以消费的产品数)
pthread_mutex_t g_mutex; //互斥信号量
pthread_t g_thread[CONSUMERS_COUNT + PRODUCERS_COUNT];
void *consume(void *arg)
{
int i;
int num = (int)arg;
while (1)
{
printf("%d wait buffer not empty\n", num);
sem_wait(&g_sem_empty);
pthread_mutex_lock(&g_mutex);
//遍历缓冲区,看有哪些缓冲区是可以生产产品的
for (i = 0; i < BUFFSIZE; i++)
{
printf("%02d ", i);
if (g_buffer[i] == -1)
printf("%s", "null");
else
printf("%d", g_buffer[i]);
if (i == out)
printf("\t<--consume");
printf("\n");
}
//produce()操作(生产产品)
consume_id = g_buffer[out];
printf("%d begin consume product %d\n", num, consume_id);
g_buffer[out] = -1;
//将取出缓冲区的指针偏移1(下个生产的位置)
out = (out + 1) % BUFFSIZE;
printf("%d end consume product %d\n", num, consume_id);
pthread_mutex_unlock(&g_mutex);
sem_post(&g_sem_full);
sleep(1);
}
return NULL;
}
void *produce(void *arg)
{
int num = (int)arg;
int i;
while (1)
{
printf("%d wait buffer not full\n", num);
sem_wait(&g_sem_full);
pthread_mutex_lock(&g_mutex);
for (i = 0; i < BUFFSIZE; i++)
{
printf("%02d ", i);
if (g_buffer[i] == -1)
printf("%s", "null");
else
printf("%d", g_buffer[i]);
if (i == in)
printf("\t<--produce");
printf("\n");
}
printf("%d begin produce product %d\n", num, produce_id);
g_buffer[in] = produce_id;
in = (in + 1) % BUFFSIZE;
printf("%d end produce product %d\n", num, produce_id++);
pthread_mutex_unlock(&g_mutex);
sem_post(&g_sem_empty);
sleep(5);
}
return NULL;
}
int main(void)
{
int i;
for (i = 0; i < BUFFSIZE; i++)
g_buffer[i] = -1;
sem_init(&g_sem_full, 0, BUFFSIZE);
sem_init(&g_sem_empty, 0, 0);
pthread_mutex_init(&g_mutex, NULL);
for (i = 0; i < CONSUMERS_COUNT; i++)
pthread_create(&g_thread[i], NULL, consume, (void *)i);
for (i = 0; i < PRODUCERS_COUNT; i++)
pthread_create(&g_thread[CONSUMERS_COUNT + i], NULL, produce, (void *)i);
for (i = 0; i < CONSUMERS_COUNT + PRODUCERS_COUNT; i++)
pthread_join(g_thread[i], NULL);
sem_destroy(&g_sem_full);
sem_destroy(&g_sem_empty);
pthread_mutex_destroy(&g_mutex);
return 0;
}