【Python爬虫】2.爬取搜狗主页搜索

import requests

query = input('输入要查询的东西')
url = f'https://www.sogou.com/web?query={query}'
hs = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"
}

response = requests.get(url, headers=hs)

page_text = response.text
print(page_text)
#step4
with open('./sogou_liujiran.html' , 'w' , encoding='utf-8') as fp:
    fp.write(page_text)
print('仿人爬取数据结束！！！')
response.close()

 

注意：

1、被反爬之后，首先看header里面的User-Agent是否有问题

2、response.close()