爬虫 code为200，但页面错误的问题

先放输出结果

E:\Python38\python.exe E:/PycharmProjects/test.py
http://www.szse.cn/api/disc/announcement/annList?random=0.16208973259833276



  
    
    
    
    
  



  

    
  


import json
import time
import datetime
import requests

t = time.time()
random = '0.' + str(t).replace(".", '')
url = "http://www.szse.cn/api/disc/announcement/annList?random=" + random
print(url)
headers = {
    "Host": "www.szse.cn",
    "Referer": "http://www.szse.cn/disclosure/bond/notice/index.html",
    "Origin": "http://www.szse.cn",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36",
}
data = {"seDate": ["", ""],
        "channelCode": ["bondinfoNotice_disc"],
        "smallCategoryId": ["013901"],
        "pageSize": 30,
        "pageNum": 1
        }
response = requests.post(url=url, headers=headers, data=json.dumps(data))
print(response)
print(response.content.decode())



Process finished with exit code 0

代码

import json
import time
import datetime
import requests

t = time.time()
random = '0.' + str(t).replace(".", '')
url = "http://www.szse.cn/api/disc/announcement/annList?random=" + random
print(url)
headers = {
    "Host": "www.szse.cn",
    "Referer": "http://www.szse.cn/disclosure/bond/notice/index.html",
    "Origin": "http://www.szse.cn",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36",
}
data = {"seDate": ["", ""],
        "channelCode": ["bondinfoNotice_disc"],
        "smallCategoryId": ["013901"],
        "pageSize": 30,
        "pageNum": 1
        }
response = requests.post(url=url, headers=headers, data=json.dumps(data))
print(response)
print(response.content.decode())

在这里我们可以看到requests请求是成功的,code为200，但返回的页面确实错误的。

经过多次尝试在不断添加headers中的参数

最后在添加"Content-Type": "application/json",之后成功获得了数据。

我想这也是一种反爬手段，你必须要去请求获得的数据类型，才能获得数据。