python的requests用法详解

Requests是一个Python语言写的http相关设置或者请求的一个库

安装：pip install Requests或者pip3 install requests

使用的时候要import requests

 

http://httpbin.org/：这个链接可以用来检查你请求的链接返回的内容，输出的是你请求的基本内容，可以用来测试验证

 

get请求

1.基本get请求的写法：

import requests

response = requests.get("http://httpbin.org/get")

print(response.text)

 

打印出你的请求头，请求链接，IP地址等

2.带参数的get请求(在链接的后面加上问号再加参数)：

Response = requests.get("http://httpbin.org/get?name=germey&age=22")

# 还有一种方式：就是先定义一个字典，然后调用get方法的时候，将字典赋值给给params参数

data = {

‘name’:’germey’,

‘age’:22

}

response = requests.get("http://httpbin.org/get",params=data)

print(response.text)

 

3.解析json

Json用来保存一些键值对组成的数据，用于数据交换，也可用于前后端之间互相传递数据，比如前端发起请求，调用接口，后端返回一串json数据，处理数据，渲染到页面上。

Request模块中也有解析json的方法：

import requests

response = requests.get("http://httpbin.org/get")

print(type(response.text))

print(response.json())

print(type(response.json()))

 

 

还可以这样写：

Import requests

Import json

response = requests.get("http://httpbin.org/get")

#和request.json()是一个意思，都是打印出一个字典数据，这个json()方法也是调用了json的loads方法

print(json.loads(requests.test()))

 

 

4.获取二进制数据

一般用来下载图片、视频等

response = requests.get("http://github.com/favicon.ico")   #将要下载的图片链接放这

print(type(response.text),type(response.content))  #类型分别是str和bytes

print(response.text)

print(response.content)     #获取二进制数据的方法

 

图片的二进制数据获取到后怎么保存呢？

import requests

response = requests.get("http://github.com/favicon.ico")

with open("favicon.ico","wb") as f:

    f.write(response.content)

    f.close()

 

5.添加headers

有时候不加headers发请求的时候会直接被拒绝或者服务器错误等，加上headers就可以了。

比如下面这段会报500错误：

response = requests.get("https://www.zhihu.com/expiore")

print(response.text)

 

 

此时需要添加一个headers：

1 headers = {
2 
3 "User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36"
4 
5 }
6 
7 response = requests.get("https://www.zhihu.com/expiore",headers=headers)
8 
9 print(response.text)

 

基本post请求，和get请求类似

6.需要的表单数据也用一个字典存起来给data参数，headers和get方法一样

Import requests

headers = {

"User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36"

}

data = {

‘name’:’germey’,

‘age’:22

}

# 需要注意和get方法不同的是如果有参数用的是data=而不是params=

response = requests.post("https://www.zhihu.com/expiore",data = data,headers = headers)

print(response.json())

 

响应

7.response属性

response = requests.post("https://www.jianshu.com")

print(type(response.status_code),response.status_code)

print(type(response.headers),response.headers)

print(type(response.cookies),response.cookies)

print(type(response.url),response.url)

print(type(response.history),response.history)   #历史记录

 

 

8.状态码判断

1 response = requests.post("https://www.jianshu.com")
2 
3 exit() if not response.status_code==200 else print("request sucessfully")
4 
5 或者可以写成：
6 
7 exit() if not response.status_code==request.codes.ok else print("request sucessfully")

 

状态码和request.codes列表中的英文状态对应表

 

状态码	英文
100	("continue",)
101	("switching_protocols")
102	("processing",)
103	("checkpoint",)
122	("url_too_long","request_url_too_long")
200	("ok","okay","all_ok","all_good",\\o/,"√")
201	("created",)
202	("accepted",)
203	("non_authoritative_info","non_authoritative_information")
204	("no_content",)
205	("reset_content","reset")
206	("partial_content","partial")
207	("multi_status","multiple_status","multi_stati","multiple_stati")
208	("already_reported",)
300	("multiple_choices",)
301	("moved_permanentiy","moved","\\o-")
302	("found",)
303	("see_other","other")
304	("not_modified")
305	("use_proxy")
306	("switch_proxy")
307	("temporary_redirect","remporary_moved","temporary")
308	("permanent_redirect","resume_incomplete","resume")
400	("bad_request","bad")
401	("unauthorized",)
402	("payment_required","payment")
403	("forbidden",)
404	("not_found","-o-")
405	("method_not_allowed","not_allowed")
406	("not_acceptable",)
407	("proxy_authentication_required","proxy_auth","proxy_authentication")
408	("request_timeout","timeout")
409	("conflict",)
410	("gone",)
411	("length_required",)
412	("precondition_falled","precondition")
413	("request_entity_too_large",)
414	("request_uril_too_large",)
415	("unsupported_media_type","unsupported_media","media_type")
416	("requested_range_not_satisfiable"," requested_range"," range_not_satisfiable")
417	("expectation_failed",)
418	("im_a_teapot","teapot","i_am_a_teapot")
421	("misdirected_request",)
422	("unprocessable_entity","unprocessable")
423	("locked",)
424	("failed_dependency","dependency")
425	("unordered_collection","unordered")
426	("upgrade_required","upgrade")
428	("precondition_required","precondition")
429	("too_many_requests","too_mary")
431	("header_fields_too_large"," fields_too_large")
444	("no_response","none")
449	("retry_with","retry")
450	("unavallable_for_legal_reasons","parental_controls")
451	("unavallable_for_legal_reasons","legal_reasons")
499	("client_closed_request",)
500	("internal_server_error","server_error","/o\\","×")
501	("not_implemented",)
502	("bad_gateway",)
503	("service_unavaliable","unavaliable")
504	("gateway_timeout")
505	("http_version_not_supported","http_version")
506	("variant_also_negotiates",)
507	("insufficient_storage",)
509	("bandwidth_limit_exceeded","bandwidth")
510	("not_extended",)
511	("network_authentication_required","network_auth","network_authentication")

 

 

 

 

request的高级操作

文件上传

1 import requests
2 
3 files = {"file":open("favicon",rb)}
4 
5 response = requests.post("http://httpbin.org/post",files=files)   #需要用post方法
6 
7 print(response.text)

 

 

text返回的files是一个文件字节流

 

获取cookie

1 response = requests.get("http://www.baidu.com")
2 
3 print(response.cookies)
4 
5 for key,value in response.cookies.items():
6 
7 print(key + "=" + value)

 

 

会话维持：用来模拟登录

1 requests.get("http://httpbin.org/cookies/set/number/1234")   #调用cookies的set方法设置cookies
2 
3 response = requests.get("http://httpbin.org/cookies")  #获取cookies
4 
5 print(response.cookies)

 

这样结果出来是空的，因为你请求了两次get方法，等于是两个浏览器各自请求

修改一下：

1 s = requests.Session()
2 
3 s.get("http://httpbin.org/cookies/set/number/1234")
4 
5 response = s.get("http://httpbin.org/cookies")
6 
7 print(response.text)

 

 

证书验证

#这个12306的证书是一个不安全的，直接请求会报错：SSL error证书错误
response = requests.get("http://www.12306.cn")   

# 这样改之后就返回200ok了，但是会有警告

 response = requests.get("http://www.12306.cn",verfiy=False)

 

再修改：

1 from requests.packages import urllib3
2 
3 urllib3.disable_warnings()
4 
5 response = requests.get("http://www.12306.cn",verfiy=False)
6 
7 print(response.text)

 

这样就没有警告了

 

代理设置

设置http或者https代理，并且没有用户名密码的情况下：

 1 Import requests
 2 
 3 proxies = {
 4 
 5 "http":"http://127.0.0.1:9743/",
 6 
 7 "https":"https://127.0.0.1:9743/"
 8 
 9 }
10 
11 response = requests.get("https://www.taobao.com",proxies=proxies)
12 
13 print(response.status_code)

 

设置http或者https代理，有用户名密码

proxies = {

"http":" http://user：password@ 127.0.0.1:9743/"

}

response = requests.get("https://www.taobao.com",proxies=proxies)

print(response.status_code)

 

如果代理不是http或者https，是socks4或者socks5，则字典的键值还是http，但是值变了：

首先需要 pip install "requests[socks]"

Import requests

proxies = {

"http":"socks5://127.0.0.1:9743/",

"https":"socks5://127.0.0.1:9743/"

}

response = requests.get("https://www.taobao.com",proxies=proxies)

print(response.status_code)

 

超时设置

import requests

response = requests.get("https://www.taobao.com",timeout=1)  #设置超过1秒就算超时

print(response.status_code)

 

超时以后会报一个异常：ReadTimeout

可以使用try捕获异常，修改代码为：

Import requests

from requests.exceptions import ReadTimeout

try：

　　response = requests.get("https://www.taobao.com",timeout=1)  #设置超过1秒就算超时

　　print(response.status_code)

except ReadTimeout：

    print("timeout")

 

认证设置

有写网址需要用户名密码登录后才可以看到内容，则：

import requests

from requests.auth import HTTPBasicAuth

r = requests.get("http://120.27.34.24:9001",auth=HTTPBasicAuth("user","123"))

print(r.status_code)

 

 

或者第二种写法：

Import requests

 

r=requests.get("http://120.27.34.24:9001",auth= ("user","123"))

print(r.status_code)

 

 

1. 异常处理

import requests

from requests.exceptions import ReadTimeout,HTTPError,ConnectionError,RequestException

try:
    response = requests.get("https://www.taobao.com",timeout=1)
    print(response.status_code)
except ReadTimeout:
    print("timeout")
except HTTPError:           #http异常
    print("http error")
except ConnectionError:       #网络不好
    print("ConnectionError")
except RequestException:    #父类异常
    print("error")