eventlet引发的学习－python：单线程、多线程在IO两方面的性能对比

测试

测试环境

CPU：Intel Core i5， 2核
磁盘：Macintosh HD

测试程序说明

* 单线程：共写入2个文件，每个文件1G
* 多线程（2个）：每个线程写入1个文件，每个文件1G
* 多进程（2个）：每个进程写入1个文件，每个文件1G
* 写入情况：每次写入文件时，字符串大小为1G，共写1次。

代码

from threading import Thread
from timeit import Timer
from multiprocessing import Process

G_COUNT = 1
G_STR_LEN = 1024 * 1024 * 1024
G_THREAD_COUNT = 2
G_PROCESS_COUNT = 2

def countdown(n):
    while n > 0:
        n -= 1

def io_opt(file_paths, n):
    tmp = n
    for file_path in file_paths:
        n = tmp
        fp = open(file_path, "w+")
        while n > 0:
            fp.write("I"* G_STR_LEN)
            n -= 1

        fp.close()

def thread_start(thread_pool):
    for thread in thread_pool:
        thread.start()

def thread_join(thread_pool):
    for thread in thread_pool:
        thread.join()

def process_start(process_pool):
    thread_start(process_pool)

def process_join(process_pool):
    thread_join(process_pool)


def single_thread():
    count = G_COUNT
    thread_pool = []
    file_paths = ["single_thread_1.txt", "single_thread_2.txt"]

    thread = Thread(target=io_opt, args=(file_paths, count,))
    thread_pool.append(thread)

    thread_start(thread_pool)
    thread_join(thread_pool)

def multi_thread():
    count = G_COUNT
    thread_pool = []
    file_paths = ["multi_thread_1.txt", "multi_thread_2.txt"]
    for i in range(G_THREAD_COUNT):
        thread = Thread(target=io_opt, args=([file_paths[i]], count,))
        thread_pool.append(thread)
    thread_start(thread_pool)
    thread_join(thread_pool)

def multi_process():
    count = G_COUNT
    process_pool = []
    file_paths = ["multi_process_1.txt", "multi_process_2.txt"]
    for i in range(G_PROCESS_COUNT):
        process = Process(target=io_opt, args=([file_paths[i]], count,))
        process_pool.append(process)

    process_start(process_pool)
    process_join(process_pool)


def main():
    t = Timer(single_thread)
    print("Single thread:%f" % (t.timeit(1),))

    t = Timer(multi_thread)
    print("Multi thread:%f" % (t.timeit(1),))

    t = Timer(multi_process)
    print("Multi process:%f" % (t.timeit(1),))

if __name__ == "__main__":
    main()

执行结果：

第1次

Single thread:9.028862
Multi thread:8.965928
Multi process:8.988812

第2次

Single thread:8.115543
Multi thread:7.932845
Multi process:8.061677

第3次

Single thread:7.944598
Multi thread:7.874828
Multi process:8.228857

分析&结论

• 通过3次结果来看，整体上多线程的更快一点，但是与单线程、多进程的差距不大

• 过程中出现了偏差较大情况，但是单线程、多线程、多进程着三者均出现过是其中的最大值，所以并未录入结果统计中，测试结果中较多的还是相差不大的内容

TODO

• 目前还不是很清楚这样测试有没有什么条件会影响
• 希望高手指导python性能相关的内容