shell 多线程脚本

shell 多线程脚本实现

方法一： for 循环 加&，指定数量的所有进程在后台同时运行

脚本示例如下，同时运行50个进程：
[root@node1 sh]# cat process.sh

#!/bin/sh

#2021年4月2日16:41:37

echo "run 50 process..."
for ((i=0;i<50;i++))
do
{
	echo "process $i is running ,sleep 5.."
	sleep 5
}&
done

wait 
echo "total time consume is $SECONDS"

说明：

1. {代码块}& 的意思是{}之间的代码全部放后台运行
2. wait 的作用是等待所有do和done之间的代码块全部运行完成后再往后执行

效果如下：
[root@node1 sh]# sh process.sh
run 50 process…
process 0 is running ,sleep 5…
process 1 is running ,sleep 5…
process 2 is running ,sleep 5…
process 3 is running ,sleep 5…
process 6 is running ,sleep 5…
process 7 is running ,sleep 5…
process 8 is running ,sleep 5…
process 9 is running ,sleep 5…
process 10 is running ,sleep 5…
process 11 is running ,sleep 5…
process 12 is running ,sleep 5…
process 5 is running ,sleep 5…
process 37 is running ,sleep 5…
process 31 is running ,sleep 5…
process 32 is running ,sleep 5…
process 4 is running ,sleep 5…
process 34 is running ,sleep 5…
process 35 is running ,sleep 5…
process 36 is running ,sleep 5…
process 33 is running ,sleep 5…
process 38 is running ,sleep 5…
process 14 is running ,sleep 5…
process 39 is running ,sleep 5…
process 15 is running ,sleep 5…
process 41 is running ,sleep 5…
process 40 is running ,sleep 5…
process 16 is running ,sleep 5…
process 17 is running ,sleep 5…
process 18 is running ,sleep 5…
process 19 is running ,sleep 5…
process 20 is running ,sleep 5…
process 21 is running ,sleep 5…
process 22 is running ,sleep 5…
process 23 is running ,sleep 5…
process 24 is running ,sleep 5…
process 25 is running ,sleep 5…
process 26 is running ,sleep 5…
process 27 is running ,sleep 5…
process 42 is running ,sleep 5…
process 28 is running ,sleep 5…
process 43 is running ,sleep 5…
process 29 is running ,sleep 5…
process 44 is running ,sleep 5…
process 30 is running ,sleep 5…
process 45 is running ,sleep 5…
process 46 is running ,sleep 5…
process 48 is running ,sleep 5…
process 13 is running ,sleep 5…
process 47 is running ,sleep 5…
process 49 is running ,sleep 5…
total time consume is 5
缺点：不可控，当后台同时运行多个进程时，cpu 内存等会消耗过多，压力过大，对于可控的运行后台进程可采用下面的方法2和3：

方法2：利用 管道+文件描述符实现控制后台同一时刻运行的进程数：

（1）知识储备

• 无名管道：ps -ef|grep nginx

• 有名管道：mkfifo /tmp/fd1
  有名管道的特效：若文件中内容为空，则读取文件会阻塞：

必须要给文件输入内容，才可以正常读取文件

这时就可以读取了：

受阻塞的echo命令：

必须有人来读文件，echo才可以进行：


由此可以看出管道文件的特点：每次只能存一个读一个，或者读一个存一个，没有则阻塞，这正是队列特性，利用这一特效可以限制shell后台进程的运行，但是管道文件只能 输入内容 -》读取文件 或者 读取文件-》输入内容这样成对的进行，问题是当往管道文件里面放入一段内容，没人取则会阻塞，这样你永远也没办法往管道里面同时放入多次（必须读-写 或写-读 顺序进行），由此引入了文件描述符。
exec 5<>/tmp/fd1，创建文件描述符5关联管道文件，这时候5这个文件描述符就拥有了管道的所有特性，还具有一个管道不具有的特性：无限存不阻塞，无限取不阻塞，而不用关心管道内是否为空，也不用关心是否有内容写入 可以执行n次echo >&5 往管道里放入n次内容，这样后面就可以执行n次 read -u5 ,当执行第n+1次 read -n5 时，才会被阻塞，这个n就可以实现控制shell后台运行的进程数。

脚本举例：

#!/bin/sh

[ -e /tmp/fd1 ]|| mkfifo /tmp/fd1
exec 5<>/tmp/fd1
rm -rf /tmp/fd1 
for i in `seq 1 10`
do
	echo ''>&5
done

for ((i=0;i<50;i++))
do
	read -u5
{
	echo "process $i is running"
	sleep 5
	echo ''>&5 
}&
done

wait
echo "total time consume is $SECONDS"
exec 5<&-
exec 5>&-

说明：

• rm -rf /tmp/fd1 文件描述符创建好后，就可以删除管道文件了
• sleep 5 模拟实际环境中进程的耗时
• exec 5<>/tmp/fd1 以读写方式创建fd1的文件描述符，即5这个文件描述符可读、可写
• exec 5<&- 关闭文件描述符的读
• exec 5>&- 关闭文件描述符的写

效果如下：
[root@node1 sh]# sh process_manage_v2.sh
process 0 is running
process 1 is running
process 6 is running
process 7 is running
process 2 is running
process 3 is running
process 9 is running
process 4 is running
process 5 is running
process 8 is running
process 10 is running
process 13 is running
process 12 is running
process 14 is running
process 15 is running
process 17 is running
process 11 is running
process 16 is running
process 18 is running
process 19 is running
process 20 is running
process 21 is running
process 22 is running
process 23 is running
process 25 is running
process 27 is running
process 29 is running
process 24 is running
process 28 is running
process 26 is running
process 30 is running
process 31 is running
process 32 is running
process 33 is running
process 34 is running
process 35 is running
process 36 is running
process 37 is running
process 38 is running
process 39 is running
process 40 is running
process 41 is running
process 42 is running
process 43 is running
process 44 is running
process 45 is running
process 46 is running
process 47 is running
process 48 is running
process 49 is running
total time consume is 25

50个进程，每次运行10个，共花费25秒

方法3：利用pid+while实现控制进程数量：

说明：当进程运行时，会创建一个/proc/pid 的目录，进程结束时，这个目录也删除了，可以用这个判断后台进程是否结束，并加上while 循环判断，当有后台进程结束时，才可以进行下一个进程的创建，否则就一直进行while循环判断，用于阻塞下一个进程的创建。while循环中加入n的限制，以保持后台进程数始终限制在n个内，示例脚本如下，实现同样的功能：

[root@node1 sh]# cat process_more.sh

#!/bin/sh
date_start=`date +%s`
i=1
max_process=10
function push_que {
que="$que  $1"
n_process=$((n_process+1))
}
function check_que {
old_que=$que
echo "now is running check_que...,old que is $old_que"
que=""
n_process=0
for pid_i in $old_que
do
	if [  -d /proc/$pid_i ];then
  		n_process=$((n_process+1))		
		que="$que $pid_i"
	fi	
done

}
for ((i=0;i<50;i++));do
{
	echo "process $i is running ,sleep 5"
	sleep 5
} &
	PID=$!
	push_que $PID
	while [[ $n_process -ge $max_process ]];do
		check_que
		sleep 0.5
		echo "now que is :$que ,n_process is $n_process"
	done 
done
wait

date_stop=`date +%s`
total_time_seconds=` expr $date_stop - $date_start  `
echo "total_time_seconds is : $total_time_seconds"
echo "total time-consuming is:"$SECONDS seconds
echo "que is $que, n_process is $n_process"

效果：
[root@node1 sh]# sh process_more.sh
process 0 is running ,sleep 5
process 1 is running ,sleep 5
process 4 is running ,sleep 5
now is running check_que…,old que is 125189 125190 125191 125192 125193 125194 125195 125196 125197 125198
process 6 is running ,sleep 5
process 7 is running ,sleep 5
process 2 is running ,sleep 5
process 9 is running ,sleep 5
process 3 is running ,sleep 5
process 5 is running ,sleep 5
process 8 is running ,sleep 5
now que is : 125191 125192 125194 125195 125196 125197 125198 ,n_process is 7
process 10 is running ,sleep 5
now is running check_que…,old que is 125191 125192 125194 125195 125196 125197 125198 125211 125212 125213
.
中间部分的日志省略…
.
now que is : 126334 126335 126336 126338 126339 126341 126342 126344 126345 126347 ,n_process is 10
now is running check_que…,old que is 126334 126335 126336 126338 126339 126341 126342 126344 126345 126347
now que is : 126334 126335 126336 126338 126339 126341 126342 126344 126345 126347 ,n_process is 10
now is running check_que…,old que is 126334 126335 126336 126338 126339 126341 126342 126344 126345 126347
now que is : ,n_process is 0
total_time_seconds is : 28
total time-consuming is:28 seconds
que is , n_process is 0
可见，每次只允许运行最大的进程数是10个，当有进程退出时才运行新的进程创建，与试验2的效果类似，花费28秒。