Mac安装scrapy的坑

Mac上的环境：OS X 10.11.6 python 2.7 & python3.5 共存

使用命令 pip3 install scrapy 后报错:

ERROR: Complete output from command python setup.py egg_info:
    ERROR: Download error on https://pypi.python.org/simple/incremental/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
    Couldn't find index page for 'incremental' (maybe misspelled?)
    Download error on https://pypi.python.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) -- Some packages may not be found!
    No local packages or download links found for incremental>=16.10.1
    Traceback (most recent call last):
      File "", line 1, in 
      File "/private/var/folders/2f/8ch08zyj75329q4x68p0lbdr0000gn/T/pip-install-opmx3gcq/Twisted/setup.py", line 20, in 
        setuptools.setup(**_setup["getSetupArgs"]())
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools/dist.py", line 269, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pkg_resources/__init__.py", line 826, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1092, in best_match
        return self.obtain(req, installer)
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1104, in obtain
        return installer(requirement)
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
        return cmd.easy_install(req)
      File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 634, in easy_install
        raise DistutilsError(msg)
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('incremental>=16.10.1')
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/2f/8ch08zyj75329q4x68p0lbdr0000gn/T/pip-install-opmx3gcq/Twisted/

报错原因:缺少用于管理python版本的 incremental 库

解决办法: 先安装incremental

pip install --upgrade incremental -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com

再次安装pip3 install scrapy 然后就提示安装成功啦

Installing collected packages: six, w3lib, PyDispatcher, attrs, pyasn1, asn1crypto, pycparser, cffi, cryptography, pyasn1-modules, service-identity, zope.interface, constantly, Automat, idna, hyperlink, PyHamcrest, Twisted, cssselect, parsel, pyOpenSSL, queuelib, scrapy
Running setup.py install for PyDispatcher … done
Running setup.py install for pycparser … done
Running setup.py install for Twisted … done
Successfully installed Automat-0.7.0 PyDispatcher-2.0.5 PyHamcrest-1.9.0 Twisted-19.2.0 asn1crypto-0.24.0 attrs-19.1.0 cffi-1.12.3 constantly-15.1.0 cryptography-2.6.1 cssselect-1.0.3 hyperlink-19.0.0 idna-2.8 parsel-1.5.1 pyOpenSSL-19.0.0 pyasn1-0.4.5 pyasn1-modules-0.2.5 pycparser-2.19 queuelib-1.5.0 scrapy-1.6.0 service-identity-18.1.0 six-1.12.0 w3lib-1.20.0 zope.interface-4.6.0

输入scrapy version -v 就显示版本以及相关环境参数信息了

Scrapy : 1.6.0
lxml : 4.3.3.0
libxml2 : 2.9.2
cssselect : 1.0.3
parsel : 1.5.1
w3lib : 1.20.0
Twisted : 19.2.0
Python : 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25) - [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
pyOpenSSL : 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019)
cryptography : 2.6.1
Platform : Darwin-15.6.0-x86_64-i386-64bit