python3 爬虫伪装headers User-Agent

在爬虫中请求中几乎都要有header请求头、fake_useragent是python中应用于UA的拓展包

安装方式   CMD中运行

pip install fake_useragent

 使用方法

from fake_useragent import UserAgent
ua = UserAgent()

print(ua.random)    #随机
print(ua.ie)        #ie
print(ua.chrome)    #Google
print(ua.firefox)    #火狐
print(ua.safari)    #苹果
print(ua.opera)        #欧朋

结果如下

Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 7.0; InfoPath.3; .NET CLR 3.1.40767; Trident/6.0; en-IN)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1468.0 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20130330 Firefox/21.0
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
Opera/12.0(Windows NT 5.1;U;en)Presto/22.9.168 Version/12.00 

实践

from fake_useragent import UserAgent
ua = UserAgent()

def html(url):
    try:
        r = req.get(url,timeout=80,headers={'User-Agent': ua.random})
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        return r.text
    except Exception as e:
        print(e)