python爬虫从入门到放弃 - requests库的使用

requests库的使用

1、requests库是什么

• requests库是基于urllib基础之上编写的，是常用的http请求库
• 安装：pip install requests

2、基本请求

req = requests.get("http://www.baidu.com")
req = requests.post("http://www.baidu.com")
req = requests.put("http://www.baidu.com")
req = requests.delete("http://www.baidu.com")
req = requests.head("http://www.baidu.com")
req = requests.options("http://www.baidu.com")

3、get请求

import requests
from fake_useragent import UserAgent

headers = {
    "User-Agent": UserAgent().chrome
}
url = "https://www.baidu.com/s"
params = {
    "wd": "尚学堂"
}
response = requests.get(url, headers=headers, params=params)
response.encoding = 'utf-8'
print(response.text)

4、post请求

import requests
from fake_useragent import UserAgent

headers = {
    "User-Agent": UserAgent().chrome
}
url = "https://www.baidu.com/s"
params = {
    "wd": "尚学堂"
}
response = requests.get(url, headers=headers, params=params)
response.encoding = 'utf-8'
print(response.text)

5、设置ip代理

import requests
from fake_useragent import UserAgent

url = "http://httpbin.org/get"
headers = {
    "User-Agent": UserAgent().chrome
}
proxies = {
    "http": "http://101.231.104.82:80"
}
response = requests.get(url, headers=headers, proxies=proxies)
print(response.text)

6、ssl验证

from fake_useragent import UserAgent
import requests

url = "https://www.12306.cn/mormhweb/"
headers = {
    "User-Agent": UserAgent().chrome
}
# 关闭警告
requests.packages.urllib3.disable_warnings()
response = requests.get(url, verify=False, headers=headers)
response.encoding = "utf-8"
print(response.text)

7、session自动保存cookie

from fake_useragent import UserAgent
import requests

session = requests.Session()
headers = {
    "User-Agent": UserAgent().chrome
}
login_url = "http://www.sxt.cn/index/login/login"
params = {
    "user": "17703181473",
    "password": "123456"
}
response = session.post(login_url, headers=headers, data=params)
info_url = "http://www.sxt.cn/index/user.html"
resp = session.get(info_url, headers=headers)
print(resp.text)

参考：https://www.bilibili.com/video/BV1z541167mu