记录 AttributeError: ‘NoneType‘ object has no attribute ‘nextcall‘

记录一次错误

scrapy 测试  scrapy bench

出现        

AttributeError: 'NoneType' object has no attribute 'nextcall'

错误大意: 某个对象没有nextcall属性

根据错误分析

Traceback (most recent call last):
  File "D:\pythoninstalllive\lib\site-packages\twisted\internet\defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "D:\pythoninstalllive\lib\site-packages\scrapy\core\engine.py", line 187, in 
    d.addBoth(lambda _: self.slot.nextcall.schedule())

找到错误发生具体位置

def _next_request_from_scheduler(self) -> Optional[Deferred]:
        assert self.slot is not None  # typing
        assert self.spider is not None  # typing

        request = self.slot.scheduler.next_request()
        if request is None:
            return None

        d = self._download(request, self.spider)
        d.addBoth(self._handle_downloader_output, request)
        d.addErrback(lambda f: logger.info('Error while handling downloader output',
                                           exc_info=failure_to_exc_info(f),
                                           extra={'spider': self.spider}))
        d.addBoth(lambda _: self.slot.remove_request(request))
        d.addErrback(lambda f: logger.info('Error while removing request from slot',
                                           exc_info=failure_to_exc_info(f),
                                           extra={'spider': self.spider}))
        # d.addBoth(lambda _: self.slot.nextcall.schedule())
        d.addErrback(lambda f: logger.info('Error while scheduling new request',
                                           exc_info=failure_to_exc_info(f),
                                           extra={'spider': self.spider}))
        return d

尝试将错误行注释, 再次运行 scrap bench

结果: 正常

分析:应该是scrapy的某个依赖库发生了变动, 将某个对象属性删除 , 索性该错误行对该程序影响不大 , 将其注释即可

你可能感兴趣的:(Web,Spider,python,爬虫,经验分享)