爬虫入门---requests初体验

好好学习,天天向上,上代码:

import requests
myheader = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"}

#请求python.org
res = requests.get("http://www.python.org",headers = myheader)
with open("p1.html","w") as file:
	file.write(res.content.decode())

#请求baidu.com
res = requests.get("https://www.baidu.com",headers = myheader)	
with open("p2.html","w",encoding = 'utf-8') as file:
	file.write(res.content.decode())

#请求163.com
res = requests.get("https://www.163.com",headers = myheader)	
with open("p3.html","w") as file:
	file.write(res.content.decode('gbk'))

• 请求python官网没有问题
• 请求baidu时,open()需要指定encoding为’utf-8’,否则报错如下:

file.write(res.content.decode())
UnicodeEncodeError: 'gbk' codec can't encode character '\xbb' in position 30252: illegal multibyte sequence

• 请求163时,decode()需要指定解码为’gbk’,否则报错如下:

file.write(res.content.decode())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcd in position 956: invalid continuation byte