Urllib库基本使用

什么是Urllib

---

Urllib是Python3内置的HTTP请求库

常用模块	释义
urllib.request	请求模块
urllib.error	异常处理模块
urllib.parse	url解析模块
urllib.robotparser	robots.txt解析模块

测试网站http://httpbin.org

例1:发送最基本的get请求,获取html代码---request.urlopen()

from urllib import request
response = request.urlopen('http://www.baidu.com')#发送请求,获取响应内容
result = response.read()#提取响应的字节流
print(result.decode('utf-8'))#将字节流转换为'utf-8',并打印

例2:post请求数据data(需要bytes类型)---parse.urlencode()

from urllib import request,parse
data = bytes(parse.urlencode({'word':'hi'}),encoding='utf-8')
response = request.urlopen('http://httpbin.org/post',data=data)
print(response.read())

例3:创造请求头,增加headers---request.Request()

from urllib import request,parse
url = 'http://httpbin.org/post'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36",
          }
dict = {
    'name':'Hello',
}
data = bytes(parse.urlencode(dict),encoding='utf-8')
req = request.Request(url=url,data=data,headers=headers,method='POST')
response = request.urlopen(req)
print(response.read().decode('utf-8'))