scrapy使用报错cannot import name ‘Deque‘

一、环境

1、python3.6.0

2、redhat 7

3、scrapy

二、问题

使用scrapy 创建新项目时，报错如下：

[root@#localhost pp]# scrapy startproject mySpider
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 5, in 
    from scrapy.cmdline import execute
  File "/usr/local/lib/python3.6/site-packages/scrapy/__init__.py", line 12, in 
    from scrapy.spiders import Spider
  File "/usr/local/lib/python3.6/site-packages/scrapy/spiders/__init__.py", line 10, in 
    from scrapy.http import Request
  File "/usr/local/lib/python3.6/site-packages/scrapy/http/__init__.py", line 8, in 
    from scrapy.http.headers import Headers
  File "/usr/local/lib/python3.6/site-packages/scrapy/http/headers.py", line 3, in 
    from scrapy.utils.python import to_unicode
  File "/usr/local/lib/python3.6/site-packages/scrapy/utils/python.py", line 16, in 
    from scrapy.utils.decorators import deprecated
  File "/usr/local/lib/python3.6/site-packages/scrapy/utils/decorators.py", line 4, in 
    from twisted.internet import defer, threads
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 32, in 
    from twisted.logger import Logger
  File "/usr/local/lib/python3.6/site-packages/twisted/logger/__init__.py", line 105, in 
    from ._logger import Logger, _loggerFor
  File "/usr/local/lib/python3.6/site-packages/twisted/logger/_logger.py", line 269, in 
    _log = Logger()
  File "/usr/local/lib/python3.6/site-packages/twisted/logger/_logger.py", line 65, in __init__
    from ._global import globalLogPublisher
  File "/usr/local/lib/python3.6/site-packages/twisted/logger/_global.py", line 17, in 
    from ._buffer import LimitedHistoryLogObserver
  File "/usr/local/lib/python3.6/site-packages/twisted/logger/_buffer.py", line 10, in 
    from typing import Deque, Optional
ImportError: cannot import name 'Deque'

参考如下文章说到：

• 原因： python > 3.6.1 才有这个方法。请升级 python

成功解决[ from typing import ( ImportError: cannot import name ‘Deque‘]_Tina姐的博客-CSDN博客ImportError: cannot import name ‘Deque’问题描述： 从 typing 里面 import Deque发生错误。原因： python > 3.6.1 才有这个方法。请升级 pythonhttps://blog.csdn.net/u014264373/article/details/119389205

三、解决：重新下载python3.7

1、安装Python-3.7.5

参考：Python安装报错：”ModuleNotFoundError:No module named _ctypes“ 的解决方案_六指黑侠i的博客-CSDN博客_modulenotfounderror解决方案

2、安装完之后要pip指定一下软连接，因为有多个pip，导致报错

[root@#localhost bin]# pip
Traceback (most recent call last):
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pip 22.2.2 (/usr/local/python3/lib/python3.7/site-packages), Requirement.parse('pip==8.0.2'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/pip", line 5, in 
    from pkg_resources import load_entry_point
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3191, in 
    @_call_aside
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3175, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3204, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/python3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==8.0.2' distribution was not found and is required by the application

ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

或用指定的python的pip版本安装包：

python -m pip install scrapy
重新安装后，可正常执行

[root@#localhost pp]# scrapy startproject tutorial
New Scrapy project 'tutorial', using template directory '/usr/local/lib/python3.7/site-packages/scrapy/templates/project', created in:
    /root/pp/tutorial

You can start your first spider with:
    cd tutorial
    scrapy genspider example example.com