爬虫遇到的错误,网上说的主要是HTTPs的ssl问题,
加一个免认证请求,应该能有限解决
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='####', port=443): Max retries exceeded with url: /chapter/OPnFZngokvlwy7E0qeXDZQ2/eosYAD10uZe2uJcMpdsVgA2/ (Caused by NewConnectionError('
1.先检查pip有没安装cryptography,pyOpenSSL,certifi要是没有先安装
pip install cryptography
pip install pyOpenSSL
pip install certifi
(我是将上面的第三方包安装好后就不再报错了)
而python的Anaconde版本中一般有包含了的。
要是以上这些已安装还是继续报错,则在网页请求代码中加上这个 verify=False 就可以解决报错,代码可运行。
我用的是requests,
那么应该这样子改写,如下:
Python3访问HTTPS时移除SSL认证:
response=requests.get(url=url,headers=headers,verify=False)
2.而至于做的话,代码虽然正常了,但是还是触发其他的小问题的:
requests库提示警告:InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
移除认证后控制台总是抛出警告:
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
为了不烦人,直接移除警告:disable_warnings()
看不见就搞定了!
文档说明如下:
在请求代码前添加如下代码即可:
import urllib3
urllib3.disable_warnings()