Python构建随机请求头headers

我的个人博客:https://www.blog.kamisamak.com/

---

layout: post
title: Python构建随机请求头headers
date: 2020-06-19
tags: [“Python”,“爬虫”]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-n2wZsw9S-1599705264908)(timg.jpg)]
通过User-Agent构建请求头headers能够将爬虫伪装，但是有的网站会去检查请求头，所以在每次提交网站请求时可以使用随机的header，频繁更换UserAgent可以避免触发相应的反爬机制，而fake-useragent对此类需求提供了解决方案用于python爬虫伪装。
通过pip进行下载

pip install fake-useragent

导库

import os
import fake_useragent
# ua = UserAgent(use_cache_server=False)
# ua = UserAgent(cache=False)
# ua = UserAgent(verify_ssl=False)

导入以下代码

# from:https://fake-useragent.herokuapp.com/browsers/0.1.11
def get_header():
    location = os.getcwd() + '/data/fake_useragent.json'
    ua = fake_useragent.UserAgent(path=location)
    # return ua.chrome
    # return ua.ie
    # return ua.firefox
    # return ua.opera
    # return ua.safari
    return ua.random

for i in range(10):
    print(get_header())

测试输出

Mozilla/5.0 (X11; CrOS i686 4319.74.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36 Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10
Mozilla/5.0 (X11; NetBSD) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0
Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.17 Safari/537.36
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36
Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36

ome/28.0.1464.0 Safari/537.36
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2117.157 Safari/537.36