2019独角兽企业重金招聘Python工程师标准>>> 
Python2和Python3的取舍
包没有安装，安装之
最近想研究一下Python，看了3和2，也敲了代码，开始敲的3，后来敲的2，感觉2和3区别不大，而且2的后续版本已经在尽力向3靠近了，而且2也更成熟，所以最终决定使用2来做研究和开发。这篇文章写的是我在安装Scrapy框架时遇到的问题。
检测Python版本为2.7，后面的版本号不做纠结，区别不大，想升级的可自行升级。
[listen@localhost ~]$ python
Python 2.7.5 (default, Sep 15 2016, 22:37:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
安装Scrapy
切换到root用户，因为有些目录普通用户没有权限
[root@localhost listen]# pip install Scrapy
如果pip工具没有安装，请先安装，可用yum install pip 进行安装，在此不表
安装到最后会报这样的错误，搞了半天也没有找到原因，就在国外的网站上找到类似的解答，就试试，没想到还真解决了
running build_ext
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c conftest.c -o conftest.o
building 'twisted.test.raiser' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/twisted
creating build/temp.linux-x86_64-2.7/src/twisted/test
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c src/twisted/test/raiser.c -o build/temp.linux-x86_64-2.7/src/twisted/test/raiser.o
src/twisted/test/raiser.c:4:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ovYzET/Twisted/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-jZG5jR-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-ovYzET/Twisted/
[root@localhost listen]# pip install python-dev
Collecting python-dev
Could not find a version that satisfies the requirement python-dev (from versions: )
No matching distribution found for python-dev
[root@localhost listen]# yum install python-devel
Loaded plugins: fastestmirror, langpacks
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
Loading mirror speeds from cached hostfile
* base: mirrors.cn99.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package python-devel.x86_64 0:2.7.5-39.el7_2 will be installed
--> Processing Dependency: python(x86-64) = 2.7.5-39.el7_2 for package: python-devel-2.7.5-39.el7_2.x86_64
--> Running transaction check
---> Package python.x86_64 0:2.7.5-34.el7 will be updated
---> Package python.x86_64 0:2.7.5-39.el7_2 will be an update
--> Processing Dependency: python-libs(x86-64) = 2.7.5-39.el7_2 for package: python-2.7.5-39.el7_2.x86_64
--> Running transaction check
---> Package python-libs.x86_64 0:2.7.5-34.el7 will be updated
---> Package python-libs.x86_64 0:2.7.5-39.el7_2 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================================
Installing:
python-devel x86_64 2.7.5-39.el7_2 updates 392 k
Updating for dependencies:
python x86_64 2.7.5-39.el7_2 updates 89 k
python-libs x86_64 2.7.5-39.el7_2 updates 5.6 M
Transaction Summary
==============================================================================================================================================================================
Install 1 Package
Upgrade ( 2 Dependent packages)
Total size: 6.1 M
Total download size: 392 k
Is this ok [y/d/N]: y
Downloading packages:
python-devel-2.7.5-39.el7_2.x86_64.rpm | 392 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : python-libs-2.7.5-39.el7_2.x86_64 1/5
Updating : python-2.7.5-39.el7_2.x86_64 2/5
Installing : python-devel-2.7.5-39.el7_2.x86_64 3/5
Cleanup : python-2.7.5-34.el7.x86_64 4/5
Cleanup : python-libs-2.7.5-34.el7.x86_64 5/5
Verifying : python-devel-2.7.5-39.el7_2.x86_64 1/5
Verifying : python-2.7.5-39.el7_2.x86_64 2/5
Verifying : python-libs-2.7.5-39.el7_2.x86_64 3/5
Verifying : python-libs-2.7.5-34.el7.x86_64 4/5
Verifying : python-2.7.5-34.el7.x86_64 5/5
Installed:
python-devel.x86_64 0:2.7.5-39.el7_2
Dependency Updated:
python.x86_64 0:2.7.5-39.el7_2 python-libs.x86_64 0:2.7.5-39.el7_2
Complete!
[root@localhost listen]# yum install python-dev
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.cn99.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
No package python-dev available.
Error: Nothing to do
[root@localhost listen]# pip install Scrapy
Requirement already satisfied: Scrapy in /usr/lib/python2.7/site-packages/Scrapy-1.2.1-py2.7.egg
Collecting Twisted>=10.0.0 (from Scrapy)
Using cached Twisted-16.5.0.tar.bz2
Collecting w3lib>=1.15.0 (from Scrapy)
Using cached w3lib-1.16.0-py2.py3-none-any.whl
Collecting queuelib (from Scrapy)
Using cached queuelib-1.4.2-py2.py3-none-any.whl
Requirement already satisfied: lxml in /usr/lib64/python2.7/site-packages (from Scrapy)
Requirement already satisfied: pyOpenSSL in /usr/lib64/python2.7/site-packages (from Scrapy)
Collecting cssselect>=0.9 (from Scrapy)
Using cached cssselect-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in /usr/lib/python2.7/site-packages (from Scrapy)
Collecting parsel>=0.9.3 (from Scrapy)
Using cached parsel-1.0.3-py2.py3-none-any.whl
Collecting PyDispatcher>=2.0.5 (from Scrapy)
Using cached PyDispatcher-2.0.5.tar.gz
Collecting service_identity (from Scrapy)
Using cached service_identity-16.0.0-py2.py3-none-any.whl
Requirement already satisfied: zope.interface>=3.6.0 in /usr/lib64/python2.7/site-packages (from Twisted>=10.0.0->Scrapy)
Requirement already satisfied: constantly>=15.1 in /usr/lib/python2.7/site-packages (from Twisted>=10.0.0->Scrapy)
Requirement already satisfied: incremental>=16.10.1 in /usr/lib/python2.7/site-packages (from Twisted>=10.0.0->Scrapy)
Collecting attrs (from service_identity->Scrapy)
Using cached attrs-16.2.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service_identity->Scrapy)
Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Requirement already satisfied: pyasn1 in /usr/lib/python2.7/site-packages (from service_identity->Scrapy)
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface>=3.6.0->Twisted>=10.0.0->Scrapy)
Installing collected packages: Twisted, w3lib, queuelib, cssselect, parsel, PyDispatcher, attrs, pyasn1-modules, service-identity
Running setup.py install for Twisted ... done
Running setup.py install for PyDispatcher ... done
Successfully installed PyDispatcher-2.0.5 Twisted-16.5.0 attrs-16.2.0 cssselect-1.0.0 parsel-1.0.3 pyasn1-modules-0.0.8 queuelib-1.4.2 service-identity-16.0.0 w3lib-1.16.0
[root@localhost listen]# python
Python 2.7.5 (default, Sep 15 2016, 22:37:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import twisted
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>>
[3]+ Stopped python
[root@localhost listen]#
安装Twisted库的时候比较费劲，下载速度特别慢，还容易超时，安装好几次才成功，安装的时候多试几次，本来就3M的包，硬是下载了几十分钟，除了Twisted包其他的包都很容易就下载安装，这就是我在安装的过程中遇到的问题，可以给大家参考一下。
测试Scrapy是否安装成功
[root@localhost listen]# scrapy startproject tutorial
Traceback (most recent call last):
File "/usr/bin/scrapy", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3011, in
parse_requirements(__requires__), Environment()
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 630, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (pyasn1 0.1.6 (/usr/lib/python2.7/site-packages), Requirement.parse('pyasn1>=0.1.8'))
报错了，说明没有安装成功，看提示说有依赖包没有安装，安装之
[root@localhost listen]# pip install parse
Collecting parse
Downloading parse-1.6.6-py2-none-any.whl
Installing collected packages: parse
Successfully installed parse-1.6.6
[root@localhost listen]#
[root@localhost listen]# scrapy startproject tutorial
Traceback (most recent call last):
File "/usr/bin/scrapy", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 3011, in
parse_requirements(__requires__), Environment()
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 630, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (pyasn1 0.1.6 (/usr/lib/python2.7/site-packages), Requirement.parse('pyasn1>=0.1.8'))
[root@localhost listen]# pip install pyasn1 --upgrade
Collecting pyasn1
Downloading pyasn1-0.1.9-py2.py3-none-any.whl
Installing collected packages: pyasn1
Found existing installation: pyasn1 0.1.6
Uninstalling pyasn1-0.1.6:
Successfully uninstalled pyasn1-0.1.6
Successfully installed pyasn1-0.1.9
[root@localhost listen]# scrapy startproject tutorial
New Scrapy project 'tutorial', using template directory '/usr/lib/python2.7/site-packages/Scrapy-1.2.1-py2.7.egg/scrapy/templates/project', created in:
/home/listen/tutorial
You can start your first spider with:
cd tutorial
scrapy genspider example example.com
[root@localhost listen]#
最后从网上找到，是需要安装pip install pyasn1 --upgrade这个东东，安装好之后就成功了。
在开发的时候肯定会需要调试，所以最好把调试工具也安装上
[root@localhost turorial]# pip install ipython
Collecting ipython
Downloading ipython-5.1.0-py2-none-any.whl (747kB)
100% |████████████████████████████████| 747kB 11kB/s
Collecting pygments (from ipython)
Downloading Pygments-2.1.3-py2.py3-none-any.whl (755kB)
100% |████████████████████████████████| 757kB 87kB/s
Collecting setuptools>=18.5 (from ipython)
Downloading setuptools-28.8.0-py2.py3-none-any.whl (472kB)
100% |████████████████████████████████| 481kB 60kB/s
Collecting backports.shutil-get-terminal-size; python_version == "2.7" (from ipython)
Downloading backports.shutil_get_terminal_size-1.0.0-py2.py3-none-any.whl
Collecting traitlets>=4.2 (from ipython)
Downloading traitlets-4.3.1-py2.py3-none-any.whl (74kB)
100% |████████████████████████████████| 81kB 26kB/s
Collecting simplegeneric>0.8 (from ipython)
Downloading simplegeneric-0.8.1.zip
Collecting pickleshare (from ipython)
Downloading pickleshare-0.7.4-py2.py3-none-any.whl
Requirement already satisfied: decorator in /usr/lib/python2.7/site-packages (from ipython)
Collecting pexpect; sys_platform != "win32" (from ipython)
Downloading pexpect-4.2.1-py2.py3-none-any.whl (55kB)
100% |████████████████████████████████| 61kB 18kB/s
Collecting prompt-toolkit<2.0.0,>=1.0.3 (from ipython)
Downloading prompt_toolkit-1.0.9-py2-none-any.whl (241kB)
100% |████████████████████████████████| 245kB 24kB/s
Collecting pathlib2; python_version == "2.7" or python_version == "3.3" (from ipython)
Downloading pathlib2-2.1.0-py2.py3-none-any.whl
Collecting ipython-genutils (from traitlets>=4.2->ipython)
Downloading ipython_genutils-0.1.0-py2.py3-none-any.whl
Requirement already satisfied: enum34; python_version == "2.7" in /usr/lib/python2.7/site-packages (from traitlets>=4.2->ipython)
Requirement already satisfied: six in /usr/lib/python2.7/site-packages (from traitlets>=4.2->ipython)
Collecting ptyprocess>=0.5 (from pexpect; sys_platform != "win32"->ipython)
Downloading ptyprocess-0.5.1-py2.py3-none-any.whl
Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.3->ipython)
Downloading wcwidth-0.1.7-py2.py3-none-any.whl
Installing collected packages: pygments, setuptools, backports.shutil-get-terminal-size, ipython-genutils, traitlets, simplegeneric, pathlib2, pickleshare, ptyprocess, pexpect, wcwidth, prompt-toolkit, ipython
Found existing installation: setuptools 0.9.8
Uninstalling setuptools-0.9.8:
Successfully uninstalled setuptools-0.9.8
Running setup.py install for simplegeneric ... done
Successfully installed backports.shutil-get-terminal-size-1.0.0 ipython-5.1.0 ipython-genutils-0.1.0 pathlib2-2.1.0 pexpect-4.2.1 pickleshare-0.7.4 prompt-toolkit-1.0.9 ptyprocess-0.5.1 pygments-2.1.3 setuptools-28.8.0 simplegeneric-0.8.1 traitlets-4.3.1 wcwidth-0.1.7
这个安装过程比较心累，发现了一个好的安装工具，可以下载试用一下Anaconda，这个工具的依赖比较全，最终的效果和手动安装一样，既然有工具为什么不用呢