python爬虫，请求获得cookies，并且携带cookies的方法

python代码：

方法一：

# 不需要登录网站，两步拿到数据
# 1.在网站首页发送请求，获取网站cookies
# 2.再次发送请求，携带cookies请求，即可得到数据

import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
# 给一个
main_url = 'https://xueqiu.com/'
response_main = requests.get(main_url, headers=headers)
print(response_main)
# 获取服务器端响应的cookie
cookies = response_main.cookies
print(cookies)
print(dict(cookies))
# 异步加载数据的url
url = 'https://xueqiu.com/statuses/hot/listV2.json?since_id=-1&max_id=366242&size=15'
response = requests.get(url, headers=headers, cookies=cookies)
print(response)

方法二：

import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
# 先访问首页 获取到cookie
session = requests.Session()   # 创建一个session对象
main_url = 'https://xueqiu.com/'
# 这里请求的目的就一个 拿到响应的cookie
session.get(url=main_url, headers=headers)

# 访问异步加载的地址 携带着cookie过去，session里面自带cookies，而且随时更新
url = 'https://xueqiu.com/statuses/hot/listV2.json?since_id=-1&max_id=366242&size=15'
res = session.get(url, headers=headers)
print(res)
print(res.json())