selenium自学之路：多线程（二）

multiprocessing模块
多进程multiprocessing模块的使用与多线程threading模块的方法类似。multiprocessing提供了本地和远程的并发性，有效地通过全局解释锁来使用进程（而不是线程）。由于GIL的存在，在CPU密集型程序中，使用多线程并不能有效地利用多核cpu的优势，因为一个解释器在同一时刻只会有一个线程在执行。所以multiprocessing模块可以充分利用硬件的多处理器来进行工作

from time import sleep,ctime
import multiprocessing

def super_player(file_,time):
    for i in range(2):
        print('Start playing: %s! %s'%(file_,ctime()))
        sleep(time)

lists={'铃儿响叮当.mp3':3,'大头儿子小头爸爸.mp4':5,'舒克和贝塔.mp4':4}
threads=[]
files=range(len(lists))

for file_ ,time in  lists.items():
    t=multiprocessing.Process(target=super_player,args=(file_,time))
    threads.append(t)

if __name__ == '__main__':
    for t in files:
        threads[t].start()
    for t in files:
        threads[t].join()
    print('end: %s'%ctime())

运行结果：

Start playing: 铃儿响叮当.mp3! Mon Apr 22 16:49:25 2019
Start playing: 大头儿子小头爸爸.mp4! Mon Apr 22 16:49:25 2019
Start playing: 舒克和贝塔.mp4! Mon Apr 22 16:49:25 2019
Start playing: 铃儿响叮当.mp3! Mon Apr 22 16:49:28 2019
Start playing: 舒克和贝塔.mp4! Mon Apr 22 16:49:29 2019
Start playing: 大头儿子小头爸爸.mp4! Mon Apr 22 16:49:30 2019
end: Mon Apr 22 16:49:35 2019

Pipe和Queue
multiprocessing提供了threading没有的IPC(进程间通信)，效率上更高。应优先考虑Pipe和Queue，避免使用Lock/Event/Semaphore/Condition等方式（因为他们占据的不是用户进程资源）
Pipe可以使单向也可以是双向，单向管道只允许管道一端的进程输入，而双向管道则允许从两端输入

import multiprocessing

def proc1(pipe):
    pipe.send('hello')
    print('proc1 rec:',pipe.recv())

def proc2(pipe):
    print('proc2 rec:',pipe.recv())
    pipe.send('hello,too')

if __name__ == '__main__':
    multiprocessing.freeze_support()
    pipe = multiprocessing.Pipe()

    p1=multiprocessing.Process(target=proc1,args=(pipe[0],))
    p2 = multiprocessing.Process(target=proc2, args=(pipe[1],))

    p1.start()
    p2.start()
    p1.join()
    p2.join()

运行结果：

proc2 rec: hello
proc1 rec: hello,too

Queue：

import multiprocessing
import os,time

def inputQ(queue):
    info=str(os.getpid()) + '(put):' + str(time.time())
    queue.put(info)

def outputQ(queue,lock):
    info=queue.get()
    lock.acquire()
    print((str(os.getpid()) + '(get):' + info))
    lock.release()

if __name__ == '__main__':
    record1 = []
    record2 = []
    lock = multiprocessing.Lock()#加锁防止乱打印
    queue = multiprocessing.Queue(3)

    for i in range(10):
        process = multiprocessing.Process(target=inputQ,args=(queue,))
        process.start()
        record1.append(process)

    for i in range(10):
        process = multiprocessing.Process(target=outputQ,args=(queue,lock))
        process.start()
        record2.append(process)

    for p in record1:
        p.join()

    queue.close()

    for p in record2:
        p.join()

运行结果：

25584(get):25124(put):1555987077.1881726
23904(get):20320(put):1555987077.2191744
9064(get):25064(put):1555987077.2531762
9128(get):15648(put):1555987077.335181
20020(get):25412(put):1555987077.4311864
24932(get):24652(put):1555987077.4981904
17940(get):24796(put):1555987077.5291922
22136(get):21184(put):1555987077.5551937
3780(get):25220(put):1555987077.630198
24848(get):9408(put):1555987077.701202

多线程执行测试用例

from threading import Thread
from selenium import webdriver
from time import ctime,sleep

def test_baidu(browser,search):
    print('start:%s'%ctime())
    print('browser:%s'%browser)
    if browser == 'ie':
        driver=webdriver.Ie()
    elif browser == 'chrome':
        driver=webdriver.Chrome()
    elif browser == 'ff':
        driver=webdriver.Firefox()
    else:
        print('browser参数有误，只能为ie,ff,chrome')

    driver.get('http://www.baidu.com')
    driver.find_element_by_id('kw').send_keys(search)
    driver.find_element_by_id('su').click()
    sleep(2)
    driver.quit()

if __name__ == '__main__':
    list={"ie":"threading","chrome":"selenium","ff":"python"}
    threads=[]
    files=range(len(list))

    for browser,search in list.items():
        t = Thread(target=test_baidu,args=(browser,search))
        threads.append(t)

    for t in files:
        threads[t].start()

    for t in files:
        threads[t].join()

    print('end:%s'%ctime())