python -- map并发

map本来就简洁

urls = ["http://www.baidu.com", "http://www.google.com.hk"]
results = map(urllib2.urlopen, urls)

和for..in..比简洁很多,把结果存到results list里

下面开始本文重点

import urllib2 
from multiprocessing.dummy import Pool as ThreadPool 

urls = [
        'http://www.python.org', 
        'http://www.python.org/about/',
        'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
        'http://www.python.org/doc/',
        'http://www.python.org/download/',
        'http://www.python.org/getit/',
        'http://www.python.org/community/',
        'https://wiki.python.org/moin/',
        'http://planet.python.org/',
        'https://wiki.python.org/moin/LocalUserGroups',
        'http://www.python.org/psf/',
        'http://docs.python.org/devguide/',
        'http://www.python.org/community/awards/'
        # etc.. 
        ]

# Make the Pool of workers
pool = ThreadPool(4) 
# Open the urls in their own threads
# and return the results
results = pool.map(urllib2.urlopen, urls)
#close the pool and wait for the work to finish 
pool.close() 
pool.join()

4句完成了并发访问

注:参照 http://www.oschina.net/translate/python-parallelism-in-one-line