多线程、多进程、池化

python程序默认单核线性执行程序，为更高效利用计算机的多核、多线程能力，python搭建了多线程、多进程的机制，更高效的完成程序。多线程模块threading、多进程模块multiprocess、池化模块concurrent.future

Corey Schafer : Python Threading Tutorial: Run Code Concurrently Using the Threading Module
莫烦Python : Multiprocessing 让你的多核计算机发挥真正潜力 Python
菜鸟教程 Python3多线程
知乎 ：python中multiprocess的参数共享与传递

threading模块、多线程，主要针对I/O更高的任务（输入/输出文件、图像等，上传/下载任务等）

import threading
import time

start = time.perf_counter()

def do_something():
    print(f'Sleeping 1 second...')
    time.sleep(1)
    print(f'Done sleeping...')

# 线性执行两个命令
# do_something()
# do_something()

# 双线程执行
# t1 = threading.Thread(target=do_something)
# t2 = threading.Thread(target=do_something)
# t1.start()
# t2.start()
# t1.join()
# t2.join()

# 更好的多线程操作
threads = []
for _ in range(10):
    t = threading.Thread(target=do_something)
    t.start()
    threads.append(t)

for thread in threads:
    thread.join()

finish = time.perf_counter()
print(f"finish in {round(finish - start,2)} second(s)")

########################################################
#结果
(base) ➜  Corey_Schafer python sp_demo.py
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Sleeping 1 second...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
Done sleeping...
finish in 1.01 second(s)

multiprocess 多进程，主要针对CPU计算任务，比如图像压缩，数值计算等

import multiprocessing
import time

start = time.perf_counter()

def do_something(secs):
    print(f'Sleeping {secs} second...\n')
    time.sleep(secs)
    print(f'Done sleeping...{secs}')

# do_something()
# do_something()

# finish = time.perf_counter()
# print(f"finish in {round(finish - start,2)} second(s)")

if __name__ == '__main__':
    processes = []
    for _ in range(4):
        p = multiprocessing.Process(target=do_something,args=[1.5])
        p.start()
        processes.append(p)

    for process in processes:
        process.join()

    finish = time.perf_counter()
    print(f"finish in {round(finish - start,2)} second(s)")
########################################################
(base) ➜  Corey_Schafer python sp_demo.py
Sleeping 1.5 second...

Sleeping 1.5 second...

Sleeping 1.5 second...

Sleeping 1.5 second...

Done sleeping...1.5
Done sleeping...1.5
Done sleeping...1.5
Done sleeping...1.5
finish in 1.62 second(s)

concurrent.futures 模块的池化操作，自动利用计算机的核，自动分配任务

import concurrent.futures
import time

start = time.perf_counter()

def do_something(secs):
    print(f'Sleeping {secs} second...\n')
    time.sleep(secs)
    return f'Done sleeping...{secs}'

if __name__ == '__main__':
    with concurrent.futures.ProcessPoolExecutor() as executor:
        secs = [4,3,2,1]
        results = executor.map(do_something,secs)

        for result in results:
            print(result)

注意⚠️

执行必须在 if name == 'main': 框架下操作，python=3.7版本以下，不需要；3.8版本之后必须要指定if name == 'main':至于为什么用，解释如下：
程序中常见的语句if name =='main': ，这段代码的含义是当该程序做为主程序启动时，执行下面的代码。对于multiprocessing的程序来说，这段代码是必不可少的，因为它提供了主进程入口，并通过主程序来启动子程序。如果没有这段代码，python无法明确判断那个是主进程，这样就无法分配子进程。