爬虫随手记

python3安装beautifulsoup4:

pip3 install beautifulsoup4

headers:
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36

安装scrapy模块出现错误:

 build/temp.linux-x86_64-3.5/_openssl.c:434:30: fatal error: openssl/opensslv.h: 没有那个文件或目录
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

解决方法:

apt-get install build-essential libssl-dev libffi-dev python3-dev

你可能感兴趣的:(爬虫随手记)