五、使用cURL发送请求

输出##一、HTTP协议组成

• target: url protocol host portx
• request
  • request: get、post、put、delete、head
  • header: host cookie user-agent
  • get query
  • post body: json xml form
• response:
  • status line
  • header

  • body

    
    
    image.png

二、Chrome开发者工具使用

1. copy as curl功能，可脱离浏览器工具发起请求

image.png

• 作用
  • 把浏览器发送的请求真实的还原出来
  • 附带了认证信息，所以可以脱离浏览器执行
  • 可以方便开发者重放请求、修改参数调试，编写脚本

三、客户端模拟请求工具

• nc tcp / udp协议发送
• curl 最常使用的http请求工具
• postman 综合性的http协议测试工具
• 代理工具、IDE工具、浏览器插件工具

1. curl常用方法

• `url=http://www.baidu.com'
• get请求curl $url
• post请求curl -d 'xxx'$url
• proxy使用 curl -x 'http://127.0.0.1:8080'$url

2. 常用参数

• -H "Content-Type: application/json”消息头设置
• -u username:password用户认证
• -d 要发送的post数据@file表示来自于文件
• --data-urlencode 'page_size=50'对内容进行url编码
• -G把data数据当成get请求的参数发送，常与--data-urlencode结合使用
• -o写文件
• -x代理http代理socks5代理
• -v verbose打印更详细日志-s 关闭一些提示输出
• --help 查看帮助文档
• jq 可以格式化输出json文件

3. 常用命令

url=http://www.baidu.com

## get请求加json解析 
curl -s 'https://xueqiu.com/stock/search.json?code=sogo&size=3&page=1' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Accept: application/json, text/plain, */*' -H 'Sec-Fetch-Dest: empty' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36' -H 'elastic-apm-traceparent: 00-760301b0a132e9a4c0f5ac7448a3419e-8823be75504fc61f-00' -H 'Sec-Fetch-Site: same-origin' -H 'Sec-Fetch-Mode: cors' -H 'Referer: https://xueqiu.com/k?q=sogo' -H 'Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7' -H 'Cookie: device_id=24700f9f1986800ab4fcc880530dd0ed; cookiesu=841584103115161; aliyungf_tc=AQAAAIPytE8aVQoAXhjf3cw3R+j5DD/s; acw_tc=2760824b15851452106833674e25941ad47588d5d7ded79b38a04dad8f9444; xq_a_token=2ee68b782d6ac072e2a24d81406dd950aacaebe3; xqat=2ee68b782d6ac072e2a24d81406dd950aacaebe3; xq_r_token=f9a2c4e43ce1340d624c8b28e3634941c48f1052; xq_id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOi0xLCJpc3MiOiJ1YyIsImV4cCI6MTU4NzUyMjY2MSwiY3RtIjoxNTg1MTQ1MTYxMDIwLCJjaWQiOiJkOWQwbjRBWnVwIn0.TPrw6_M2Th9QTVz5spwUybqN1790nJANu9kxXl4GfNb1eQ2p2zD43CStgogOGQ8yRXYmSCfURp0343wgjnnCdnQX5698Jl-brdP94wiYKwv11q8QjBYMXFWJGRj0g69C2nxVrRF8K-ETGEked3KjYfk8Xy2wPuZtyGUhORWeCvMhmBdcRKIlWj4d7wp-w_LjMbSLigJAT29F03wBZIxR0r3eMNUhUsXh8dCsWNb6wzhtg8dT4gcd91mQmR5ToR_SFrzQfOopY4vQGcaOHWaAwUMPLUopZwD4ajWzm1kpoBZnf_n_9uBfT4j0nGk95E8J8EmTfBlq-1p019xkhgp87w; u=431585145210698; Hm_lvt_1db88642e346389874251b5a1eded6e3=1583285031,1584102200,1585145180; Hm_lpvt_1db88642e346389874251b5a1eded6e3=1585145192' --compressed | jq


# 输出结果
{
  "q": "sogo",
  "page": 1,
  "size": 3,
  "stocks": [
    {
      "code": "SOGO",
      "name": "搜狗",
      "enName": "",
      "hasexist": "false",
      "flag": null,
      "type": 0,
      "stock_id": 1029472,
      "ind_id": 0,
      "ind_name": "通讯业务",
      "ind_color": null,
      "_source": "sc_1:1:sogo"
    }
  ]
}


#post请求
curl 'http://sonarqube.testing-studio.com:9000/api/authentication/login' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Accept: application/json' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: http://sonarqube.testing-studio.com:9000' -H 'Referer: http://sonarqube.testing-studio.com:9000/sessions/new' -H 'Accept-Language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7' -H 'Cookie: _ga=GA1.2.232181868.1566982077; experimentation_subject_id=IjNlYzgxODQ1LTU2MDAtNGIyNy1iNTgzLTE1MzRkY2IwMDI0ZSI%3D--b1f29d33f6a2c85a81be66e4774d437f710c102f; _gid=GA1.2.482544306.1585051015' --data 'login=admin&password=1234' --compressed --insecure

#百度的一个url提交脚本
curl -H 'Content-Type:text/plain' --data-binary @urls.txt "http://data.zz.baidu.com/urls"

#对参数编码并发送get请求
    curl -G $url  \
        --data-urlencode "current=$current" \
        --data-urlencode "pageSize=$pageSize" 


#认证与put上传都ElasticSearch里
    curl -X PUT "$ES_HOST/$index/_doc/$id?pretty" \
        --user username:password \
        -H 'Content-Type: application/json' \
        -d "$content"


#查看邮箱
curl -s --user $mail_username:$mail_password "imaps://imap.exmail.qq.com/inbox?all"

4. 练习

• 手工向百度发起一个http请求

  
  
  image.png

• 使用curl发起一个自定义个人署名的header信息

  
  
  image.png

下一节：常用代理工具，Charles、BurpSuite、Mitmproxy、Anyproxy等。