python3.5使用scrapy的坑

环境win10-64+python3.5+scrapy1.2
a

from scrapy.linkextractors.sgml import SgmlLinkExtractor as sle
需要改为
from scrapy.linkextractors.lxmlhtml import LxmlLinkExtractor as sle
因为sgmllib在python3中已经被删除了

b

import urlparse
要改为
import urllib.parse as urlparse
因为在python3中重构了

c

urllib.urlencode改为urllib.parse.urlencode

d

import httplib
要改为
import http.client as httplib

e
如果scrapy crawl xxx运行爬虫后出现结果报错：

Paste_Image.png

ImportError: cannot import name '_win32stdio'
ImportError: No module named 'win32api'

因为twisted需要安装依赖模块pywin32

pip install pypiwin32

最后完美运行

Paste_Image.png