Python爬虫--Rquestes

Requests官方文档：Requests官方文档

1. 安装requests

pip install requests

注意：需要安装在python解释器相同的位置,例如：D:\Program Files\Python3.11.4\Scripts

2. 导入：

 import requests

3. 基本使用

import requests

url = "https://www.baidu.com"

response = requests.get(url=url)

# 一个类型和六个属性

# response类型为 Response
print(type(response))

# 设置响应的编码格式
response.encoding = "utf-8"

# 以字符串的形式返回网页的源码
print(response.text)

# 返回url地址
print(response.url)

# 返回二进制数据
print(response.content)

# 返回响应状态码
print(response.status_code)

# 返回响应头信息
print(response.headers)

4. get请求

# 导包
import requests

# url地址
# url = "https://www.baidu.com/s？"
url = "https://www.baidu.com/s"

# 请求头
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}

# 参数
data = {
    "wd": "北京"
}

# 参数使用params传递，参数不需要使用urlencode编码
# requests请求不需要请求对象的定制
# 请求资源路径中的？可以加也可以不加
# 发起请求
response = requests.get(url=url, params=data, headers=headers)

# 获取响应
content = response.text
print(content)

5. post请求

# 导包
import requests

# url
url = "https://fanyi.baidu.com/?sug"

# 请求头
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}

# 参数
data = {
    "kw": "eye"
}

# 参数使用data传递，参数不需要使用urlencode编码
# requests请求不需要请求对象的定制
# 发起请求
response = requests.post(url=url, data=data, headers=headers)

# 获取响应
content = response.text
print(content)

6. 代理

import requests

url = "https://www.baidu.com/s?"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
data = {
    "wd": "ip"
}
# 代理
proxy = {
    "http": "121.230.210.31:3256"
}
# proxies：代理
response = requests.get(url=url, params=data, headers=headers, proxies=proxy)
content = response.text
with open("daili.html", "w", encoding="utf-8") as fp:
    fp.write(content)

7. cookie登录 

下集在见.........