使用单进程、多进程、多线程、协程测试数字货币交易所的连接速度

一、概念

1、进程
进程是具有一定独立功能的程序关于某个数据集合上的一次运行活动,进程是系统进行资源分配和调度的一个独立单位。每个进程都有自己的独立内存空间，不同进程通过进程间通信来通信。由于进程比较重量，占据独立的内存，所以上下文进程间的切换开销（栈、寄存器、虚拟内存、文件句柄等）比较大，但相对比较稳定安全。
2、线程
线程是进程的一个实体,是CPU调度和分派的基本单位,它是比进程更小的能独立运行的基本单位.线程自己基本上不拥有系统资源,只拥有一点在运行中必不可少的资源(如程序计数器,一组寄存器和栈),但是它可与同属一个进程的其他的线程共享进程所拥有的全部资源。线程间通信主要通过共享内存，上下文切换很快，资源开销较少，但相比进程不够稳定容易丢失数据。
3、协程
协程是一种用户态的轻量级线程，协程的调度完全由用户控制。协程拥有自己的寄存器上下文和栈。协程调度切换时，将寄存器上下文和栈保存到其他地方，在切回来的时候，恢复先前保存的寄存器上下文和栈，直接操作栈则基本没有内核切换的开销，可以不加锁的访问全局变量，所以上下文的切换非常快。

二、实战

交易所列表定义
需要测试的交易所有
BIKI, BINANCE, BIONE, BITFINEX, BITFLYER, BITHUMB, BITMEX, BITSTAMP, BITTREX, BITZ, CEO, COINBASE, COINEX, FCOIN, GATEIO, HUOBI, KRAKEN, MXC, OKEX, POLONIEX, WBF, XT, ZB, ZBG
定义如下

trees = ["biki","binance","bione","bitfinex","bitflyer","bithumb","bitmex","bitstamp","bittrex","bitz","ceo","coinbase","coinex","fcoin","gateio","huobi","kraken","mxc","okex","poloniex","wbf","xt","zb","zbg"]

2.1、使用进程

其实就是串行的方式运行

print('%s 开始 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
for fruit in trees:
    exchange = Exchange(fruit)
    exchange.debug = True
    exchange.ping()
print('%s 结束 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

返回结果如下

2020-05-03 21:18:53 开始
2020-05-03 21:19:13       biki      openapi.biki.com    None
2020-05-03 21:19:33    binance       api.binance.com    None
2020-05-03 21:19:40      bione          www.bione.cc    367
2020-05-03 21:20:00   bitfinex      api.bitfinex.com    None
2020-05-03 21:20:20   bitflyer      api.bitflyer.com    None
2020-05-03 21:20:40    bithumb       api.bithumb.com    None
2020-05-03 21:20:59     bitmex        www.bitmex.com    None
2020-05-03 21:21:02   bitstamp      www.bitstamp.net    74
2020-05-03 21:21:21    bittrex       api.bittrex.com    None
2020-05-03 21:21:41       bitz        apiv2.bitz.com    None
2020-05-03 21:21:44        ceo         api.ceobi.com    127
2020-05-03 21:21:56   coinbase  api.pro.coinbase.com    378
2020-05-03 21:22:07     coinex        api.coinex.com    389
2020-05-03 21:22:29      fcoin         api.fcoin.com    None
2020-05-03 21:22:49     gateio        data.gateio.io    None
2020-05-03 21:23:18      huobi         api.huobi.pro    None
2020-05-03 21:23:22     kraken        api.kraken.com    155
2020-05-03 21:23:41        mxc           www.mxc.com    None
2020-05-03 21:24:09       okex          www.okex.com    None
2020-05-03 21:24:29   poloniex          poloniex.com    None
2020-05-03 21:24:58        wbf      openapi.wbf.info    None
2020-05-03 21:25:01         xt          kline.xt.com    68
2020-05-03 21:25:30         zb             api.zb.cn    None
2020-05-03 21:25:33        zbg           www.zbg.fun    70
2020-05-03 21:25:33 结束

耗时将近7分钟

2.2、使用多进程

from multiprocessing import Process
from multiprocessing import Pool

def task(exname):
    print('%s %s is running %s' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), os.getpid(), exname))
    exchange = Exchange(exname)
    exchange.debug = True
    ttl = exchange.ping()
    print('%s %s is done %s' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), os.getpid(),exname) )
    return ttl

if __name__ == '__main__':
    print('%s 开始 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
    p=Pool(24)
    res_l=[]
 
    for ex in trees:
        res=p.apply_async(task,args=(ex,)) 
        res_l.append([ex, res])
  
    p.close()
    p.join()
    for res in res_l:
        print([res[0], res[1].get()])
    print('%s 结束 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

开12个进程时返回如下

2020-05-03 21:20:14 开始
2020-05-03 21:20:19 5912 is running biki
2020-05-03 21:20:19 2848 is running binance
2020-05-03 21:20:19 8284 is running bione
2020-05-03 21:20:19 4932 is running bitfinex
2020-05-03 21:20:19 7496 is running bitflyer
2020-05-03 21:20:20 9088 is running bithumb
2020-05-03 21:20:20 6576 is running bitmex
2020-05-03 21:20:20 6136 is running bitstamp
2020-05-03 21:20:21 2888 is running bittrex
2020-05-03 21:20:21 3428 is running bitz
2020-05-03 21:20:21 6248 is running ceo
2020-05-03 21:20:21 8332 is running coinbase
2020-05-03 21:20:24 6136 is done bitstamp
2020-05-03 21:20:24 6136 is running coinex
2020-05-03 21:20:24 6248 is done ceo
2020-05-03 21:20:24 6248 is running fcoin
2020-05-03 21:20:27 8284 is done bione
2020-05-03 21:20:27 8284 is running gateio
2020-05-03 21:20:31 6136 is done coinex
2020-05-03 21:20:31 6136 is running huobi
2020-05-03 21:20:36 8332 is done coinbase
2020-05-03 21:20:36 8332 is running kraken
2020-05-03 21:20:39 4932 is done bitfinex
2020-05-03 21:20:39 4932 is running mxc
2020-05-03 21:20:39 7496 is done bitflyer
2020-05-03 21:20:39 7496 is running okex
2020-05-03 21:20:39 8332 is done kraken
2020-05-03 21:20:39 8332 is running poloniex
2020-05-03 21:20:40 2888 is done bittrex
2020-05-03 21:20:40 9088 is done bithumb
2020-05-03 21:20:40 9088 is running xt
2020-05-03 21:20:40 5912 is done biki
2020-05-03 21:20:40 5912 is running zb
2020-05-03 21:20:40 6576 is done bitmex
2020-05-03 21:20:40 6576 is running zbg
2020-05-03 21:20:40 2888 is running wbf
2020-05-03 21:20:40 2848 is done binance
2020-05-03 21:20:42 3428 is done bitz
2020-05-03 21:20:43 6576 is done zbg
2020-05-03 21:20:43 9088 is done xt
2020-05-03 21:20:43 6248 is done fcoin
2020-05-03 21:20:48 8284 is done gateio
2020-05-03 21:20:59 4932 is done mxc
2020-05-03 21:20:59 5912 is done zb
2020-05-03 21:21:01 6136 is done huobi
2020-05-03 21:21:01 2888 is done wbf
2020-05-03 21:21:08 7496 is done okex
2020-05-03 21:21:08 8332 is done poloniex
['biki', None]
['binance', None]
['bione', 354]
['bitfinex', None]
['bitflyer', None]
['bithumb', None]
['bitmex', None]
['bitstamp', 65]
['bittrex', None]
['bitz', None]
['ceo', 66]
['coinbase', 338]
['coinex', 376]
['fcoin', None]
['gateio', None]
['huobi', None]
['kraken', 158]
['mxc', None]
['okex', None]
['poloniex', None]
['wbf', None]
['xt', 76]
['zb', None]
['zbg', 68]
2020-05-03 21:21:09 结束

消耗时间118秒，近2分钟

开24个进程时返回如下

2020-05-03 21:16:43 开始
2020-05-03 21:16:51 8304 is running biki
2020-05-03 21:16:51 3584 is running binance
2020-05-03 21:16:51 9068 is running bione
2020-05-03 21:16:51 5236 is running bitfinex
2020-05-03 21:16:51 7352 is running bitflyer
2020-05-03 21:16:51 7936 is running bithumb
2020-05-03 21:16:51 8920 is running bitmex
2020-05-03 21:16:52 3508 is running bitstamp
2020-05-03 21:16:52 1124 is running bittrex
2020-05-03 21:16:53 3504 is running bitz
2020-05-03 21:16:53 500 is running ceo
2020-05-03 21:16:53 1376 is running coinbase
2020-05-03 21:16:53 5264 is running coinex
2020-05-03 21:16:53 1648 is running fcoin
2020-05-03 21:16:55 2452 is running gateio
2020-05-03 21:16:55 6092 is running huobi
2020-05-03 21:16:55 8808 is running kraken
2020-05-03 21:16:56 7340 is running mxc
2020-05-03 21:16:56 8480 is running okex
2020-05-03 21:16:56 5680 is running poloniex
2020-05-03 21:16:56 9208 is running wbf
2020-05-03 21:16:56 3168 is running xt
2020-05-03 21:16:56 4648 is running zb
2020-05-03 21:16:57 5136 is running zbg
2020-05-03 21:16:59 3508 is done bitstamp
2020-05-03 21:17:00 500 is done ceo
2020-05-03 21:17:00 3168 is done xt
2020-05-03 21:17:00 5136 is done zbg
2020-05-03 21:17:01 8808 is done kraken
2020-05-03 21:17:04 7352 is done bitflyer
2020-05-03 21:17:04 1376 is done coinbase
2020-05-03 21:17:05 5264 is done coinex
2020-05-03 21:17:11 9068 is done bione
2020-05-03 21:17:16 1124 is done bittrex
2020-05-03 21:17:16 3584 is done binance
2020-05-03 21:17:16 1648 is done fcoin
2020-05-03 21:17:16 9208 is done wbf
2020-05-03 21:17:16 5680 is done poloniex
2020-05-03 21:17:16 5236 is done bitfinex
2020-05-03 21:17:17 8304 is done biki
2020-05-03 21:17:17 2452 is done gateio
2020-05-03 21:17:17 7340 is done mxc
2020-05-03 21:17:17 4648 is done zb
2020-05-03 21:17:22 8920 is done bitmex
2020-05-03 21:17:25 7936 is done bithumb
2020-05-03 21:17:26 6092 is done huobi
2020-05-03 21:17:26 3504 is done bitz
2020-05-03 21:17:26 8480 is done okex
['biki', None]
['binance', None]
['bione', 356]
['bitfinex', None]
['bitflyer', 186]
['bithumb', None]
['bitmex', None]
['bitstamp', 60]
['bittrex', None]
['bitz', None]
['ceo', 65]
['coinbase', 371]
['coinex', 371]
['fcoin', None]
['gateio', None]
['huobi', None]
['kraken', 141]
['mxc', None]
['okex', None]
['poloniex', None]
['wbf', None]
['xt', 70]
['zb', None]
['zbg', 59]
2020-05-03 21:17:27 结束

耗时44秒，不到1分钟

2.3、使用多线程

def task(exname):
    print('%s  %s start ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), exname))
    exchange = Exchange(exname)
    exchange.debug = True
    exchange.ping()
    # print('%s  %s end' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), exname))

print('%s 开始 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
for ex in trees:
    t = threading.Thread(target=task,args=(ex,))
    t.start()

返回如下

2020-05-03 21:30:24 开始
2020-05-03 21:30:24  biki start
2020-05-03 21:30:24  binance start
2020-05-03 21:30:24  bione start
2020-05-03 21:30:24  bitfinex start
2020-05-03 21:30:24  bitflyer start
2020-05-03 21:30:24  bithumb start
2020-05-03 21:30:24  bitmex start
2020-05-03 21:30:24  bitstamp start
2020-05-03 21:30:24  bittrex start
2020-05-03 21:30:24  bitz start
2020-05-03 21:30:24  ceo start
2020-05-03 21:30:24  coinbase start
2020-05-03 21:30:24  coinex start
2020-05-03 21:30:24  fcoin start
2020-05-03 21:30:24  gateio start
2020-05-03 21:30:24  huobi start
2020-05-03 21:30:24  kraken start
2020-05-03 21:30:24  mxc start
2020-05-03 21:30:24  okex start
2020-05-03 21:30:24  poloniex start
2020-05-03 21:30:24  wbf start
2020-05-03 21:30:24  xt start
2020-05-03 21:30:24  zb start
2020-05-03 21:30:24  zbg start
2020-05-03 21:30:28   bitstamp      www.bitstamp.net    66
2020-05-03 21:30:28        zbg           www.zbg.fun    68
2020-05-03 21:30:28      bione          www.bione.cc    363
2020-05-03 21:30:28     kraken        api.kraken.com    154
2020-05-03 21:30:28         xt          kline.xt.com    71
2020-05-03 21:30:29        ceo         api.ceobi.com    76
2020-05-03 21:30:32     coinex        api.coinex.com    363
2020-05-03 21:30:32   coinbase  api.pro.coinbase.com    370
2020-05-03 21:30:40       bitz        apiv2.bitz.com    374
2020-05-03 21:30:44     bitmex        www.bitmex.com    None
2020-05-03 21:30:44        mxc           www.mxc.com    None
2020-05-03 21:30:44    bittrex       api.bittrex.com    None
2020-05-03 21:30:44        wbf      openapi.wbf.info    None
2020-05-03 21:30:44   bitflyer      api.bitflyer.com    None
2020-05-03 21:30:44       biki      openapi.biki.com    None
2020-05-03 21:30:45         zb             api.zb.cn    None
2020-05-03 21:30:45   poloniex          poloniex.com    None
2020-05-03 21:30:45     gateio        data.gateio.io    None
2020-05-03 21:30:46   bitfinex      api.bitfinex.com    None
2020-05-03 21:30:52      fcoin         api.fcoin.com    None
2020-05-03 21:30:53    binance       api.binance.com    None
2020-05-03 21:30:54      huobi         api.huobi.pro    None
2020-05-03 21:30:54    bithumb       api.bithumb.com    None
2020-05-03 21:30:55       okex          www.okex.com    None

耗时31秒

2.4、使用协程

print('%s 开始 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))
loop = asyncio.get_event_loop()
tasks = []
for fruit in trees:
    exchange = Exchange(fruit)
    exchange.debug = True
    tasks.append(exchange.ping_async())
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
print('%s 结束 ' %(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))

返回如下

2020-05-03 21:28:24 开始
2020-05-03 21:28:27        zbg           www.zbg.fun    61
2020-05-03 21:28:27     kraken        api.kraken.com    164
2020-05-03 21:28:28   bitstamp      www.bitstamp.net    64
2020-05-03 21:28:28        ceo         api.ceobi.com    73
2020-05-03 21:28:28         xt          kline.xt.com    65
2020-05-03 21:28:31      huobi         api.huobi.pro    375
2020-05-03 21:28:31   coinbase  api.pro.coinbase.com    360
2020-05-03 21:28:35      bione          www.bione.cc    363
2020-05-03 21:28:35     coinex        api.coinex.com    361
2020-05-03 21:28:43    bittrex       api.bittrex.com    None
2020-05-03 21:28:43       biki      openapi.biki.com    None
2020-05-03 21:28:43      fcoin         api.fcoin.com    None
2020-05-03 21:28:43   poloniex          poloniex.com    None
2020-05-03 21:28:43       bitz        apiv2.bitz.com    None
2020-05-03 21:28:43   bitflyer      api.bitflyer.com    None
2020-05-03 21:28:43     bitmex        www.bitmex.com    None
2020-05-03 21:28:43    bithumb       api.bithumb.com    None
2020-05-03 21:28:43        mxc           www.mxc.com    None
2020-05-03 21:28:43   bitfinex      api.bitfinex.com    None
2020-05-03 21:28:43        wbf      openapi.wbf.info    None
2020-05-03 21:28:45    binance       api.binance.com    None
2020-05-03 21:28:47       okex          www.okex.com    None
2020-05-03 21:28:54     gateio        data.gateio.io    None
2020-05-03 21:28:54         zb             api.zb.cn    None
2020-05-03 21:28:54 结束

耗时30秒

三、总结

方式	耗时（秒）
单线程单进程	330
多进程(12个)	118
多进程(24个)	44
多线程	31
协程	30

由此可见，多线程和协程速度最快。