Python爬虫实战:使用代理(requests模块)

对于requests来说,代理设置加简单,我们只需要传入proxies参数即可

import requests

url = 'http://ip.tool.chinaz.com/'

headers = {
     
    'Host': "ip.tool.chinaz.com",
    'User-Agent': "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
}
proxy = '58.240.53.196:8080'

proxy_support = {
     
    'http':'http://'+proxy,
    'https':'https://'+proxy,
}

response = requests.get(url,proxies=proxy_support)
print(response.text)

Python爬虫实战:使用代理(requests模块)_第1张图片如果代理需要认证,同样在代理的前面加上用户名密码即可

proxies='username:[email protected]:9743'

你可能感兴趣的:(Python,爬虫学习,Python爬虫实战)