fake-useragent 能够提供随机的User-Agent,模拟浏览器使用的User-Agent。
环境:Win10系统能,Anaconda3 + PyCharm, Python3.6
一、安装步骤:
1.以管理员身份运行【Anaconda Prompt】:
2.输入命令:
pip install fake-useragent
结果如上图所示,安装成功,但是注意用 conda install fake-useragent 找不到这个包,会安装失败。
二、测试:
1.打开PyCharm编译器,新建一个文件,输入以下代码:
# 伪装User-Agent的测试
from fake_useragent import UserAgent
ua = UserAgent()
print(ua.ie)
print(ua.opera)
print(ua.chrome)
print(ua.google)
print(ua['google chrome'])
print(ua.firefox)
print(ua.ff)
print(ua.safari)
print(ua.random) #最好的, 通过真实世界的浏览器使用统计随机
2.测试结果:第一次运行提示出了问题,再次运行就OK。
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)
Opera/9.80 (Windows NT 6.1 x64; U; en) Presto/2.7.62 Version/11.00
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1664.3 Safari/537.36
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; sv-se) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
3.简单使用:
from fake_useragent import UserAgent
import requests
ua = UserAgent()
headers = {'User-Agent': ua.random}
url = 'https://www.baidu.com/'
r = requests.get(url, headers=headers)
print(r.status_code)
print(r.encoding)
print(r.apparent_encoding)
print(r.request.headers)
4.结果:
200
ISO-8859-1
utf-8
{'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.62 Safari/537.36', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
注意:在使用fake_useragent包时可能造成不能够使用的状况,最好是收集好UseAgent的列表,从中随机选择一个来使用。
参考:
https://blog.csdn.net/zhangmoyan9527/article/details/82894295
https://blog.csdn.net/qq_35531549/article/details/88364196
https://www.jianshu.com/p/74bce9140934
http://www.chenxm.cc/article/50.html