爬虫练习中遇到中文乱码的问题（记录）

代码如下

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

打开test2.txt，发现中文乱码

中文乱码.png

我们在res后面加一行代码看下返回内容的编码方式

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
print(res.encoding)
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

输出：

ISO-8859-1

第一种解决方法：

encode('iso-8859-1') 是将gbk编码编码成unicode编码
decode(‘gbk’) 是从unicode编码解码成gbk字符串

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
res = res.text.encode('iso-8859-1').decode('gbk')
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res)

第二种解决方法

# -*- coding:UTF-8 -*-
import requests

url = 'http://www.chyxx.com/industry/201711/585988.html'

res = requests.get(url)
res.encoding = 'gbk'
with open("test2.txt", 'w', encoding='utf-8') as file:
    file.write(res.text)

感谢https://blog.csdn.net/xfyangle/article/details/60969522