scrapy报错解决[twisted.internet.error.TimeoutError: User timeout caused connection failure:]

在scrapy中遇到报错：twisted.internet.error.TimeoutError: User timeout caused connection failure:

一般是设置了DOWNLOAD_TIMEOUT  之后，用了代理ip等等，就会出现这类报错。

解决方法为：

在middleware中，捕获这个报错，并返回request，让他重新请求这个对象

先导入  

from twisted.internet.error import TimeoutError

    def process_exception(self, request, exception, spider):
        # Called when a download handler or a process_request()
        # (from other downloader middleware) raises an exception.

        # Must either:
        # - return None: continue processing this exception
        # - return a Response object: stops process_exception() chain
        # - return a Request object: stops process_exception() chain
        if isinstance(exception,TimeoutError):
            return request