python进程、线程、协程
进程（Process）是计算机中的程序关于某数据集合上的一次运行活动，是系统进行资源分配和调度的基本单位，是操作系统结构的基础。在早期面向进程设计的计算机结构中，进程是程序的基本执行实体；在当代面向线程设计的计算机结构中，进程是线程的容器。程序是指令、数据及其组织形式的描述，进程是程序的实体。
线程，有时被称为轻量级进程(Lightweight Process，LWP），是程序执行流的最小单元。一个标准的线程由线程ID，当前指令指针(PC），寄存器集合和堆栈组成。另外，线程是进程中的一个实体，是被系统独立调度和分派的基本单位，线程自己不拥有系统资源，只拥有一点儿在运行中必不可少的资源，但它可与同属一个进程的其它线程共享进程所拥有的全部资源。一个线程可以创建和撤消另一个线程，同一进程中的多个线程之间可以并发执行。由于线程之间的相互制约，致使线程在运行中呈现出间断性。线程也有就绪、阻塞和运行三种基本状态。就绪状态是指线程具备运行的所有条件，逻辑上可以运行，在等待处理机；运行状态是指线程占有处理机正在运行；阻塞状态是指线程在等待一个事件（如某个信号量），逻辑上不可执行。每一个程序都至少有一个线程，若程序只有一个线程，那就是程序本身。
协程与子例程一样，协程（coroutine）也是一种程序组件。相对子例程而言，协程更为一般和灵活，但在实践中使用没有子例程那样广泛。协程源自 Simula 和 Modula-2 语言，但也有其他语言支持。协程不是进程或线程，其执行过程更类似于子例程，或者说不带返回值的函数调用。
- 进程
from multiprocessing import Process
def show(args):
print("process" + str(args))
if __name__ == "__main__":
for i in range(50):
P = Process(target=show, args=(i,))
P.start()- 进程共享数据
from multiprocessing import Process, Array, Value,Manager
def f(n, a):
n.value = 3.1415927
for i in range(len(a)):
a[i] = -a[i]
if __name__ == '__main__':
#The 'd' and 'i' arguments used when creating num and arr are typecodes of the kind used by the array module: 'd' indicates a double precision float and 'i' indicates a signed integer. These shared objects will be process and thread-safe.
num = Value('d', 0.0)
arr = Array('i', range(10))
p = Process(target=f, args=(num, arr))
p.start()
p.join()
print(num.value)
print(arr[:])from multiprocessing import Process, Manager
def f(d, l):
d[1] = '1'
d['2'] = 2
d[0.25] = None
l.reverse()
if __name__ == '__main__':
with Manager() as manager:
d = manager.dict()
l = manager.list(range(10))
p = Process(target=f, args=(d, l))
p.start()
p.join()
print(d)
print(l)- 进程池
#The Pool class represents a pool of worker processes. It has methods which allows tasks to be offloaded to the worker processes in a few different ways.
from multiprocessing import Process,Pool
def Func(i):
return i+100
def Back(args):
print(args)
pool = Pool(5)
if __name__ == "__main__":
for i in range(100,1000,100):
pool.apply_async(func=Func, args=(i,),callback=Back)
print('end')
pool.close()
pool.join()
apply(func[, args[, kwds]])
Call func with arguments args and keyword arguments kwds. It blocks until the result is ready. Given this blocks, apply_async() is better suited for performing work in parallel. Additionally, func is only executed in one of the workers of the pool.
apply_async(func[, args[, kwds[, callback[, error_callback]]]])
A variant of the apply() method which returns a result object.
If callback is specified then it should be a callable which accepts a single argument. When the result becomes ready callback is applied to it, that is unless the call failed, in which case the error_callback is applied instead.
If error_callback is specified then it should be a callable which accepts a single argument. If the target function fails, then the error_callback is called with the exception instance.
Callbacks should complete immediately since otherwise the thread which handles the results will get blocked.
- 线程
import threading
import time
def show(args):
# time.sleep(2)
print("thread" + str(args))
if __name__ == "__main__":
start = time.time()
for i in range(50):
#不使用线程所需时间为0.002998
# show(i)
#使用线程所需时间为0.014995
T = threading.Thread(target=show, args=(i,))#传参以元组形式
T.start()#开启线程
T.join()#会等待所有线程执行完毕
end = time.time()
print("所需时间为%f"% (end - start))
print("End of main thread")start：线程准备就绪，等待CPU调度
setName：为线程设置名称
getName：获取线程名称
setDaemon ：设置为后台线程或前台线程（默认）如果是后台线程，主线程执行过程中，后台线程也在进行，主线程执行完毕后，后台线程不论成功与否，均停止
如果是前台线程，主线程执行过程中，前台线程也在进行，主线程执行完毕后，等待前台线程也执行完成后，程序停止
join：逐个执行每个线程，执行完毕后继续往下执行，该方法使得多线程变得无意义
run：线程被cpu调度后自动执行线程对象的run方法
更多方法，我将上传到我的文件进行下载
- 线程同步应用
import threading
import time
global_num = 10
lock1 = threading.Lock()
lock2 = threading.Lock()
lock2.acquire()
def test1():
if lock1.acquire():
global global_num
global_num += 10
time.sleep(1)
print(global_num)
print(1)
lock2.release()
def test2():
if lock2.acquire():
global global_num
global_num += 10
time.sleep(1)
print(global_num)
print(2)
lock1.release()
if __name__ == "__main__":
for i in range(10):
t1 = threading.Thread(target=test1)
t2 = threading.Thread(target=test2)
t1.start()
t2.start()Event Objects
This is one of the simplest mechanisms for communication between threads: one thread signals an event and other threads wait for it.An event object manages an internal flag that can be set to true with the set() method and reset to false with the clear() method. The wait() method blocks until the flag is true.
is_set()
Return true if and only if the internal flag is true.
set()
Set the internal flag to true. All threads waiting for it to become true are awakened. Threads that call wait() once the flag is true will not block at all.
clear()
Reset the internal flag to false. Subsequently, threads calling wait() will block until set() is called to set the internal flag to true again.
wait(timeout=None)
Block until the internal flag is true. If the internal flag is true on entry, return immediately. Otherwise, block until another thread calls set() to set the flag to true, or until the optional timeout occurs.
When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof).
This method returns true if and only if the internal flag has been set to true, either before the wait call or after the wait starts, so it will always return True except if a timeout is given and the operation times out.
Changed in version 3.1: Previously, the method always returned None
import threading
import sys
def test(E):
print('start')
E.wait()
print('On-line Transaction Processing')
if __name__ == "__main__":
event_object = threading.Event()
for i in range(10):
t = threading.Thread(target=test, args=(event_object,))
t.start()
event_object.clear()
inp = input('input-:')
if inp == "1":
event_object.set()协程
greenlet
from greenlet import greenlet
def test1():
print(11)
two.switch()
print(22)
two.switch()
def test2():
print(33)
one.switch()
print(44)
one = greenlet(test1)
two = greenlet(test2)
one.switch()- gevent
import gevent
def func1():
print('Run this func1')
gevent.sleep(0)
print('Dynamic switching func1')
def func2():
print('Run this func2')
gevent.sleep(0)
print('Dynamic switching func2')
gevent.joinall([
gevent.spawn(func1),
gevent.spawn(func2),
])- 遇到IO操作自动切换
import gevent
import urllib.request as urllib
def func(url):
print('GET: %s' % url)
response = urllib.urlopen(url)
data = response.read()
print('%d byte comes from that %s.' % (len(data), url))
gevent.joinall([
gevent.spawn(func, 'http://blog.csdn.net/boyun58/'),
gevent.spawn(func, 'https://www.baidu.com/'),
gevent.spawn(func, 'https://www.aliyun.com/'),
])- 线程池
import queue
import threading
class ThreadPool(object):
def __init__(self, max_num=20):
print("队列仓库初始化，默认20")
self.queue = queue.Queue(max_num)
for i in range(max_num):
self.queue.put(threading.Thread)
def get_thread(self):
num = 0
while True:
if self.queue.qsize() >= 1:
num += 1
print("Clothe Sold" + str(self.queue.get()) + str(num))
else:
break
def add_thread(self):
"Add Merchandise"
num = 0
while True:
if self.queue.qsize() < 20:
self.queue.put(threading.Thread)
num += 1
print("goods" + str(num) + "successfully added")
else:
break
if __name__ == "__main__":
QT = ThreadPool()
QT.get_thread()
QT.add_thread()The asynchronous execution can be performed with threads, using ThreadPoolExecutor, or separate processes, using ProcessPoolExecutor. Both implement the same interface, which is defined by the abstract Executor class.
ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously
from concurrent.futures import ThreadPoolExecutor
import time
def func(args):
time.sleep(2)
print(args)
pool=ThreadPoolExecutor(8)
for i in range(30):
pool.submit(func,i)
submit(fn, *args, **kwargs)
Schedules the callable, fn, to be executed as fn(*args **kwargs) and returns a Future object representing the execution of the callable
map(func, *iterables, timeout=None, chunksize=1)
Equivalent to map(func, *iterables) except func is executed asynchronously and several calls to func may be made concurrently. The returned iterator raises a concurrent.futures.TimeoutError if next() is called and the result isn’t available after timeout seconds from the original call to Executor.map(). timeout can be an int or a float. If timeout is not specified or None, there is no limit to the wait time. If a call raises an exception, then that exception will be raised when its value is retrieved from the iterator. When using ProcessPoolExecutor, this method chops iterables into a number of chunks which it submits to the pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer. For very long iterables, using a large value for chunksize can significantly improve performance compared to the default size of 1. With ThreadPoolExecutor, chunksize has no effect.
shutdown(wait=True)
Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to Executor.submit() and Executor.map() made after shutdown will raise RuntimeError.