【HTTP】【HTTP协议分享】
HTTP协议分享
- 定义
- HTTP协议组成
- 请求报文 request
- 请求结构如图
- 请求方法
- GET
- POST
- HEAD
- PUT
- DELETE
- OPTIONS
- TRACE
- 响应报文 response
- 响应结构
- 响应码
- 1xx
- 2xx
- 200 OK
- 206 Partial Content
- 3xx
- 301 永久重定向
- 302 临时重定向
- 304 Not Modified
- 307 Temporary Redirect
- 4xx
- 405 Method Not Allowed
- 416 Requested Range Not Satisfiable
- 5xx
- 常用用法
- 防盗链+数据统计 - Referer
- 防盗链
- 数据统计
- 断点续传+多线程下载 - Range
督促自己学习总结，特用文章的形式记录下来，共同进步
定义
HTTP(HyperText Transfer Protocol)超文本传输协议(超文本转移协议)
HTTP是一种不保存状态，即无状态协议(状态保存引入Cookie)
HTTP是应用层协议
Web网页(如：购物网站、博客) 开发常用各种api，android中OkHttp框架
置灰后的效果
HTTP协议组成
请求报文 request
请求结构如图
POST /blog HTTP/1.1
Host: localhost:3434
User-Agent: curl/7.63.0
Accept: */*
Content-Length: 14
Content-Type: application/x-www-form-urlencoded
name=zy&age=28
简单一个请求报文请求行，请求头部和请求实体
请求方法
GET
telnet baidu.com 80
GET / HTTP/1.1
Host: www.baidu.com
curl -v www.baidu.com
分别用telnet 和curl 发送GET请求
POST
POST /blog HTTP/1.1
Host: localhost:3434
Content-Length: 93
Content-Type: application/x-www-form-urlencoded
{"id":1,"date":"2020-02-22 03:29:13","author":"fish","title":"http2","content":"http test 2"}
curl -v -X POST --data "{\"id\":2,\"date\":\"2020-02-22 03:29:13\",\"author\":\"fish\",\"title\":\"http2\",\"content\":\"http test 2\"}" http://localhost:3434/blog/1
HEAD
只返回响应头，不返回实体
telnet baidu.com 80
HEAD / HTTP/1.1
Host: www.baidu.com
Connection: Close
curl -v -X HEAD www.baidu.com
curl -v -I www.baidu.com
curl -v --head www.baidu.com
PUT
curl -v -X PUT --data "{\"id\":2,\"date\":\"2020-02-22 03:29:13\",\"author\":\"fish\",\"title\":\"http2\",\"content\":\"http test 2\"}" http://localhost:3434/blog/1
curl -v -X PUT --data-ascii "{\"id\":2,\"date\":\"2020-02-22 03:29:13\",\"author\":\"fish\",\"title\":\"http2\",\"content\":\"http test 2\"}" http://localhost:3434
DELETE
curl -v -X DELETE http://localhost:3434/blog
OPTIONS
curl -X OPTIONS https://developer.mozilla.org/ -i
curl -X OPTIONS example.org -i
telnet example.org 80
OPTIONS / HTTP/1.1
Host: example.org
86183@DESKTOP-MKB5ERF MINGW64 ~/Desktop
$ curl -X OPTIONS example.org -i
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
Allow: OPTIONS, GET, HEAD, POST
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Fri, 05 Jun 2020 00:39:33 GMT
Expires: Fri, 12 Jun 2020 00:39:33 GMT
Server: EOS (vny/0453)
Content-Length: 0
支持4中方法：Allow: OPTIONS, GET, HEAD, POST
TRACE
curl -v -X TRACE www.baidu.com -H "Max-Forwards: 2"
还有CONNECT，PATCH等
响应报文 response
响应结构
HTTP/1.1 302 Found
Connection: keep-alive
Content-Security-Policy-Report-Only: default-src https: 'unsafe-inline' 'unsafe-eval' data: blob: ; report-uri https://reports.baidu.com/csp-report/baike
Content-Type: text/html; charset=UTF-8
Date: Fri, 05 Jun 2020 00:07:24 GMT
Location: //baike.baidu.com/item/curl
Server: Apache
Content-Length: 0
响应码
1xx
信息，服务器收到请求，需要请求者继续执行操作
范例待补充
2xx
请求成功
200 OK
请求成功
curl -v -I http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg
curl -I http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 247k 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
Date: Fri, 05 Jun 2020 00:45:26 GMT
Content-Type: image/jpg
Content-Length: 253354
Connection: keep-alive
Expires: Mon, 31 May 2021 00:45:17 GMT
Server: Apache
Content-Transfer-Encoding: binary
Cache-Control: max-age=31104000
Vary: Accept-Encoding
X-Ser: BC75_dx-lt-yd-zhejiang-jinhua-5-cache-10, BC144_dx-guangdong-jiangmen-7-cache-5
X-Cache: HIT from BC144_dx-guangdong-jiangmen-7-cache-5(baishan)
206 Partial Content
发送Range的请求 请求成功会返回206请求码
curl http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg -v -H "Range: bytes=0-1000"
curl http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg -v -H "Range: bytes=0-50, 100-150"
86183@DESKTOP-MKB5ERF MINGW64 ~/Desktop
$ curl http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg -H "Range: bytes=0-1000" -v
* STATE: INIT => CONNECT handle 0x288e150; line 1392 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x288e150; line 1428 (connection #0)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 183.47.233.138...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x288e150; line 1509 (connection #0)
* Connected to a4.att.hudong.com (183.47.233.138) port 80 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x288e150; line 1561 (connection #0)
* Marked for [keep alive]: HTTP default
* STATE: SENDPROTOCONNECT => DO handle 0x288e150; line 1579 (connection #0)
> GET /21/09/01200000026352136359091694357.jpg HTTP/1.1
> Host: a4.att.hudong.com
> User-Agent: curl/7.58.0
> Accept: */*
> Range: bytes=0-1000
>
* STATE: DO => DO_DONE handle 0x288e150; line 1658 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x288e150; line 1783 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x288e150; line 1799 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 206 Partial Content
< Date: Fri, 05 Jun 2020 00:49:40 GMT
< Content-Type: image/jpg
< Content-Length: 1001
< Connection: keep-alive
< Expires: Mon, 31 May 2021 00:46:31 GMT
* Server Apache is not blacklisted
< Server: Apache
< Content-Transfer-Encoding: binary
< Cache-Control: max-age=31104000
< Vary: Accept-Encoding
< Content-Range: bytes 0-1000/253354
< X-Ser: BC75_dx-lt-yd-zhejiang-jinhua-5-cache-10, BC144_dx-guangdong-jiangmen-7-cache-5
< X-Cache: HIT from BC144_dx-guangdong-jiangmen-7-cache-5(baishan)
3xx
重定向，需要进一步的操作以完成请求
301 永久重定向
范例待补充
302 临时重定向
curl -X OPTIONS www.baidu.com -I
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 17931
Content-Type: text/html
Date: Sun, 07 Jun 2020 01:58:52 GMT
Etag: "54d9748e-460b"
Server: bfe/1.0.8.18
304 Not Modified
访问的资源未被修改，使用缓存就OK
curl 'http://ask.android-studio.org/uploads/article/20150602/49e627d8b62d6c877c805ded9c2ec964.jpg' -I
HTTP/1.1 200 OK
Date: Sun, 07 Jun 2020 01:40:09 GMT
Server: Apache/2.4.10 (Unix) OpenSSL/1.0.1j PHP/5.5.19 mod_perl/2.0.8-dev Perl/v5.16.3
Last-Modified: Tue, 02 Jun 2015 14:24:22 GMT
ETag: "ba74-51789b2d7d180"
Accept-Ranges: bytes
Content-Length: 47732
Content-Type: image/jpeg
拿到ETag:"ba74-51789b2d7d180" 在请求的头部加上该Tag字段
curl 'http://ask.android-studio.org/uploads/article/20150602/49e627d8b62d6c877c805ded9c2ec964.jpg' -H 'If-None-Match: "ba74-51789b2d7d180"' -I
HTTP/1.1 304 Not Modified
Date: Sun, 07 Jun 2020 01:41:41 GMT
Server: Apache/2.4.10 (Unix) OpenSSL/1.0.1j PHP/5.5.19 mod_perl/2.0.8-dev Perl/v5.16.3
ETag: "ba74-51789b2d7d180"
307 Temporary Redirect
临时重定向与302一样
curl -X OPTIONS developer.mozilla.org -I
HTTP/1.1 307 Temporary Redirect
Server: CloudFront
Date: Sun, 07 Jun 2020 01:46:02 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://developer.mozilla.org/
X-Cache: Redirect from cloudfront
Via: 1.1 9e763d54b60e5dbe2c1faa8e75e52b67.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: NRT12-C3
X-Amz-Cf-Id: WsIx0xTqkkZuiQy3Ce3bK9y-Uij6cSmDp0MZxJSF4b_BVjfcz_DXJQ==
这种也可以直接让命令帮你执行重定向后的地址
curl -X OPTIONS developer.mozilla.org -I -L
HTTP/1.1 307 Temporary Redirect
Server: CloudFront
Date: Sun, 07 Jun 2020 01:47:57 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://developer.mozilla.org/
X-Cache: Redirect from cloudfront
Via: 1.1 263337573333cdedec0f11e424dd369d.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: NRT12-C3
X-Amz-Cf-Id: 4luQJXzVYxKWtaCPtEyCsOyhnBN_phk_tDYTUwdKZNtQfbMxTq1vnQ==
HTTP/2 301
content-type: text/html; charset=utf-8
content-length: 0
cache-control: public, max-age=0, s-maxage=300
date: Sun, 07 Jun 2020 01:47:58 GMT
location: /en-US/
server: meinheld/1.0.1
strict-transport-security: max-age=63072000
vary: Cookie
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
x-cache: Miss from cloudfront
via: 1.1 0fde9b863333aeec4c82b26429369a68.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT12-C3
x-amz-cf-id: -uitrYvfee2XWrskMrTc4gW7AZHSx3lxPS_SVvSfl57KuZW7G9PpDA==
HTTP/2 200
content-type: text/html; charset=utf-8
content-length: 43024
vary: Accept-Encoding
cache-control: public, max-age=0, s-maxage=300
content-language: en-US
date: Sun, 07 Jun 2020 01:47:58 GMT
server: meinheld/1.0.1
strict-transport-security: max-age=63072000
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
x-cache: Miss from cloudfront
via: 1.1 0fde9b863333aeec4c82b26429369a68.cloudfront.net (CloudFront)
x-amz-cf-pop: NRT12-C3
x-amz-cf-id: PoQnt7oC9HwvYcXWC1DwdFZiTLAVZBHzZ2yxMmCeZ-UhS5YCNudYTA==
4xx
客户端错误，访问了服务端不支持方法或者不存在的资源
405 Method Not Allowed
请求的方法服务器不支持
curl -X DELETE http://example.org -i
HTTP/1.1 405 Method Not Allowed
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Sun, 07 Jun 2020 01:56:17 GMT
Expires: Sun, 14 Jun 2020 01:56:17 GMT
Server: EOS (vny/044F)
Content-Length: 0
416 Requested Range Not Satisfiable
当请求头中带有Range字段，范围合法返回206 ，范围不合法返回416
curl http://a4.att.hudong.com/21/09/01200000026352136359091694357.jpg -H "Range: bytes=253354-2533540" -v
* STATE: INIT => CONNECT handle 0xd3e150; line 1392 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0xd3e150; line 1428 (connection #0)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 183.47.233.135...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0xd3e150; line 1509 (connection #0)
* Connected to a4.att.hudong.com (183.47.233.135) port 80 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0xd3e150; line 1561 (connection #0)
* Marked for [keep alive]: HTTP default
* STATE: SENDPROTOCONNECT => DO handle 0xd3e150; line 1579 (connection #0)
> GET /21/09/01200000026352136359091694357.jpg HTTP/1.1
> Host: a4.att.hudong.com
> User-Agent: curl/7.58.0
> Accept: */*
> Range: bytes=253354-2533540
>
* STATE: DO => DO_DONE handle 0xd3e150; line 1658 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0xd3e150; line 1783 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0xd3e150; line 1799 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 416 Requested Range Not Satisfiable
< Date: Sun, 07 Jun 2020 02:04:13 GMT
< Content-Type: text/html
< Content-Length: 234
< Connection: keep-alive
* Server web cache is not blacklisted
< Server: web cache
< Expires: Sun, 07 Jun 2020 02:04:13 GMT
< X-Ser: BC144_dx-guangdong-jiangmen-7-cache-5
< X-Cache: HIT from BC144_dx-guangdong-jiangmen-7-cache-5(baishan)
5xx
服务器内部错误
常用用法
防盗链+数据统计 - Referer
Referer: https://www.baidu.com/?tn=88093251_41_hao_pg
Referer: https://www.baidu.com/more/
referer: https://v.qq.com/
referer: https://v.qq.com/channel/usuk
防盗链
Referer是指网页是谁发起的请求，是从哪个页面发起的
通过Referer字段可以屏蔽非法的请求来源，范例：QQ空间的图片，如果非法就返回一个default的图片
当然这种方式是不准确的，我们可以自己构造合法的参数绕过服务器检测
数据统计
Referer是指网页是谁发起的请求，是从哪个页面发起的
很好理解，服务器统计这些数据是用来统计流量来源，统计占比，分析流量入口变化情况，从而指定运营策略和方向等等
比如某个来源流量上升了，提升了10%，那么大数据就要分析为什么升了，是由哪些运营策略影响的，还是有新的需求灰度，或者说分析这个数据在灰度期间是不是正常的。
同理某个渠道的来源下降了，那么也要分析原因，是运营策略造成的，还是说代码存在BUG等，这些都是需要分析的
断点续传+多线程下载 - Range
对于文件可以在请求头中添加Range字段，告诉服务器我只请求某些范围的数据，从而实现断点续传和多线程下载功能。
这个算是任何下载功能模块的的基本功能