python异步处理请求_使用Python请求的异步请求

注意

以下答案不适用于请求v0.13.0。此问题写入后，异步功能已移至grequests。但是，你可以只是替换请求与grequests下面，它应该工作。

我离开这个答案是为了反映原来的问题是关于使用请求< v0.13.0。 要使用async.map异步执行多个任务，您必须：

>为每个对象(你的任务)定义一个函数，

>将此函数作为事件挂钩添加到请求中

>在所有请求/操作的列表上调用async.map

例：

from requests import async

# If using requests > v0.13.0, use

# from grequests import async

urls = [

'http://python-requests.org',

'http://httpbin.org',

'http://python-guide.org',

'http://kennethreitz.com'

]

# A simple task to do to each response object

def do_something(response):

print response.url

# A list to hold our things to do via async

async_list = []

for u in urls:

# The "hooks = {..." part is where you define what you want to do

#

# Note the lack of parentheses following do_something, this is

# because the response will be used as the first argument automatically

action_item = async.get(u, hooks = {'response' : do_something})

# Add the task to our list of things to do via async

async_list.append(action_item)

# Do our list of things to do via async

async.map(async_list)