curl命令
curl是利用URL语法在命令行方式下工作的文件传输工具
它支持很多协议：FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE 以及 LDAP
cURL is a computer software project providing alibrary and command-line tool for transferring data using variousprotocols.The cURL project produces two products,libcurl and cURL. It wasfirst released in 1997.
libcurl
A free client-side URL transfer library, supporting FTP,FTPS,Gopher, HTTP, HTTPS, SCP, SFTP, TFTP, Telnet, DICT, the file URI scheme,LDAP, LDAPS,IMAP, POP3, SMTP and RTSP. The library supports HTTPS certificates,HTTP POST,HTTP PUT, FTP uploading,Kerberos, HTTP form based upload, proxies,cookies, user-plus-password authentication, file transfer resume, andHTTP proxy tunneling.
The libcurl library is portable. It builds and works identically on several platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HPUX, IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Symbian, Amiga, OS/2, BeOS, Mac OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS and more.
The libcurl library is free, thread-safe, IPv6 compatible, and fast.Bindings in more than 40languages are available for
curl
它支持很多协议：FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE 以及 LDAP。 curl同样支持HTTPS认证，HTTP POST方法, HTTP PUT方法, FTP上传, kerberos认证, HTTP上传, 代理服务器, cookies, 用户名/密码认证, 下载文件断点续传, 上载文件断点续传, http代理服务器管道（ proxy tunneling）, 甚至它还支持IPv6, socks5代理服务器, 通过http代理服务器上传文件到 FTP服务器等等，功能十分强大。
Windows 操作系统下的网络蚂蚁， 网际快车(FlashGet)的功能它都可以做到。准确的说，curl支持文件的上传和下载，所以是一个综合传输工具，但是按照传统，用户习惯称curl为 下载工具。
curl命令用法
获得页面
使用命令：curl http://curl.haxx.se 这是最简单的使用方法。用这个命令获得了http://curl.haxx.se指向的页面，同样，如果这里的URL指向的是一个文件或者一幅图都可以直接下载到本地。如果下载的是HTML文档，那么缺省的将不显示文件头部，即HTML文档的header。
####(试过好像是显示<head>部分的啊???， 应该是curl的 输出 把 http响应的报头部分省略了，而只显示实体部分， 此时如果 用把 -i -O 选项一起用的话，会把报头也写到文件里去）
要全部显示，请加参数 -i， （我的测试结果是 使用i会把 http response 的body 部分也显示？）
要只显示头部，用参数 -I。
任何时候，可以使用 -v 命令看curl是怎样工作的，它显示出与服务器连接的过程，并把请求和相应的报头也显示出来。
为了断点续传，可以使用-r参数来指定传输范围。
获取表单
在WEB页面设计中，form是很重要的元素。Form通常用来收集并向网站提交信息。提交信息的方法有两种，GET方法和POST方法。先讨论GET方法，例如在页面中有这样一段： <form method="GET" action="junk.cgi"> <input type=text name="birthyear"> <input type=submit name=press value="OK"> </form> 那么 浏览器上会出现一个文本框和一个标为“OK”的按钮。按下这个按钮，表单就用GET方法向服务器提交文本框的数据。 例如原始页面是在 www.hotmail. com/when/birth.html看到的，然后您在文本框中输入1905，然后按OK按钮，那么浏览器的URL现在应该是：“www.hotmail. com/when/junk.cgi?birthyear=1905&press=OK” 对于这种网页，curl可以直接处理，例如想获取上面的网页，只要输入： curl "www.hotmail. com/when/junk.cgi?birthyear=1905&press=OK" 就可以了。 表单用来提交信息的第二种方法叫做POST方法，POST方法和GET方法的区别在于GET方法使用的时候，浏览器中会产生目标URL，而POST不会。类似GET，这里有一个网页： <form method="POST" action="junk.cgi"> <input type=text name="birthyear"> <input type=submit name=press value="OK"> </form> 浏览器上也会出现一个文本框和一个标为“OK”的按钮。按下这个按钮，表单用POST方法向服务器提交数据。 这时的URL是看不到的，因此需要使用特殊的方法来抓取这个页面： curl -d "birthyear=1905&press=OK" www.hotmail. com/when/junk.cgi 这个命令就可以做到。 1995年年末，RFC 1867定义了一种新的POST方法，用来上传文件。主要用于把本地文件上传到服务器。此时页面是这样写的： <form method="POST" enctype='multipart/form-data' action="upload.cgi"> <input type=file name=upload> <input type=submit name=press value="OK"> </form> 对于这种页面，curl的用法不同： curl -F upload=@localfilename -F press=OK URL 这个命令的实质是将本地的文件用POST上传到服务器。 有关POST还有不少用法，用户可以自己摸索。使用PUT
HTTP协议文件上传的标准方法是使用PUT，此时curl命令使用-T参数： curl -T uploadfile www.uploadhttp. com/receive.cgi有关认证
curl可以处理各种情况的认证页面，例如下载用户名/密码认证方式的页面（在IE中通常是出现一个输入用户名和密码的输入框）： curl -u name:password www.secrets. com 如果网络是通过http代理服务器出去的，而代理服务器需要用户名和密码，那么输入： curl -U proxyuser:proxypassword http://curl.haxx. se 任何需要输入用户名和密码的时候，只在参数中指定用户名而空着密码，curl可以交互式的让用户输入密码。引用
有些 网络资源访问的时候必须经过另外一个网络地址跳转过去，这用术语来说是：referer，引用。 对于这种地址的资源，curl也可以下载： curl -e http://curl.haxx. se daniel.haxx. se指定用户端
有些网络资源首先需要判断用户使用的是什么浏览器，符合标准了才能够下载或者浏览。 此时curl可以把自己“伪装”成任何其他浏览器： curl -A "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" URL 这个指令表示curl伪装成了IE5.0，用户平台是Windows 2000。（对方服务器是根据这个字串来判断 客户端的类型的，所以即使使用AIX也无所谓）。 使用： curl -A "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" URL 此时curl变成了Netscape，运行在PIII平台的Linux上了。COOKIES
Cookie是服务器经常使用的一种记忆客户信息的方法。如果cookie被记录在了文件中，那么使用命令： curl -b stored_cookies_in_file www.cookiesite. com curl可以根据旧的cookie写出新cookie并发送到网站： curl -b cookies.txt -c newcookies.txt www.cookiesite. com加密HTTP
如果是通过OpenSSL加密的https协议传输的网页，curl可以直接访问： curl https://that.secure.server. comhttp认证
如果是采用证书认证的http地址，证书在本地，那么curl这样使用： curl -E mycert.pem https://that.secure.server. com注意事项
curl非常博大，用户要想使用好这个工具，除了详细学习参数之外，还需要深刻理解http的各种协议与URL的各个语法。 这里推荐几个读物： RFC 2616 HTTP协议语法的定义。 RFC 2396 URL语法的定义。 RFC 2109 Cookie是怎样工作的。 RFC 1867 HTTP如何POST，以及POST的格式。使用举例
[root@fsc01~]# curl http://en.wikipedia.org/wiki/CURL
%Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
041192 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
[root@fsc01 scratch]# curl http://en.wikipedia.org/wiki/CURL | less
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 41192 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en" dir="ltr" class="client-nojs" xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>cURL - Wikipedia, the free encyclopedia</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="generator" content="MediaWiki 1.18wmf1" />
<link rel="alternate" type="application/x-wiki" title="Edit this page" href="/w/index.php?title=CURL&action=edit" />
<link rel="edit" title="Edit this page" href="/w/index.php?title=CURL&action=edit" />
<link rel="apple-touch-icon" href="//en.wikipedia.org/apple-touch-icon.png" />
<link rel="shortcut icon" href="/favicon.ico" />
<link rel="search" type="application/opensearchdescription+xml" href="/w/opensearch_desc.php" title="Wikipedia (en)" />
<link rel="EditURI" type="application/rsd+xml" href="//en.wikipedia.org/w/api.php?action=rsd" />
<link rel="copyright" href="//creativecommons.org/licenses/by-sa/3.0/" />
<link rel="alternate" type="application/atom+xml" title="Wikipedia Atom feed" href="/w/index.php?title=Special:RecentChanges&feed=atom" />
<link rel="stylesheet" href="//bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.wikihiero%7Cmediawiki.legacy.commonPrint%2Cshared%7Cskins.vector&only=styles&skin=vector&*" type="text/css" media="all" />
<meta name="ResourceLoaderDynamicStyles" content="" />
<link rel="stylesheet" href="//bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=site&only=styles&skin=vector&*" type="text/css" media="all" />
<style type="text/css" media="all">a:lang(ar),a:lang(ckb),a:lang(fa),a:lang(kk-arab),a:lang(mzn),a:lang(ps),a:lang(ur){text-decoration:none}a.new,#quickbar a.new{color:#ba0000}
/* cache key: enwiki:resourceloader:filter:minify-css:4:c88e2bcd56513749bec09a7e29cb3ffa */</style>
<script src="//bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=startup&only=scripts&skin=vector&*" type="text/javascript"></script>
<script type="text/javascript">if ( window.mediaWiki ) {
mw.config.set({"wgCanonicalNamespace": "", "wgCanonicalSpecialPageName": false, "wgNamespaceNumber": 0, "wgPageName": "CURL", "wgTitle": "CURL", "wgCurRevisionId": 462386479, "wgArticleId": 519497, "wgIsArticle": true, "wgAction": "view", "wgUserName": null, "wgUserGroups": ["*"], "wgCategories": ["Articles needing additional references from November 2010", "All articles needing additional references", "Articles lacking reliable references from August 2010", "All articles lacking reliable references", "Free web crawlers", "Download managers", "Free FTP clients", "HTTP clients", "Free cross-platform software", "Free software programmed in C", "C libraries", "1997 software", "Software using the MIT license"], "wgBreakFrames": false, "wgRestrictionEdit": [], "wgRestrictionMove": [], "wgSearchNamespaces": [0], "wgFlaggedRevsParams": {"tags": {"status": {"levels": 1, "quality": 2, "pristine": 3}}}, "wgStableRevisionId": null, "wgVectorEnabledModules": {"collapsiblenav": true, "collapsibletabs": true, "editwarning": true, "expandablesearch": false, "footercleanup": false, "sectioneditlinks": false, "simplesearch": true, "experiments": true}, "wgWikiEditorEnabledModules": {"toolbar": true, "dialogs": true, "hidesig": true, "templateEditor": false, "templates": false, "preview": false, "previewDialog": false, "publish": false, "toc": false}, "wgTrackingToken": "2890394d06d9ff273688
100 41192 100 41192 0 0 22545 0 0:00:01 0:00:01 --:--:-- 33930
31e95a0f8df9", "wikilove-recipient": "", "wikilove-edittoken": "+\\", "wikilove-anon": 0, "mbEditToken": "+\\", "Geo": {"city": "", "country": ""}, "wgNotic
eProject": "wikipedia"});
}
</script><script type="text/javascript">if ( window.mediaWiki ) {
mw.loader.load(["mediawiki.page.startup"]);
}
</script>
<!--[if lt IE 7]><style type="text/css">body{behavior:url("/w/skins-1.18/vector/csshover.min.htc")}</style><![endif]--></head>
<body class="mediawiki ltr sitedir-ltr ns-0 ns-subject page-CURL action-view skin-vector">
<div id="mw-page-base" class="noprint"></div>
<div id="mw-head-base" class="noprint"></div>
<!-- content -->
<div id="content">
<a id="top"></a>
<div id="mw-js-message" style="display:none;"></div>
<!-- sitenotice -->
<div id="siteNotice"><!-- centralNotice loads here --></div>
。。。。。。
加上-v选项
[root@fsc01 scratch]# curl --url http://en.wikipedia.org/wiki/CURL -v | less
* About to connect() to en.wikipedia.org port 80
* Trying 208.80.152.201... connected
* Connected to en.wikipedia.org (208.80.152.201) port 80
> GET /wiki/CURL HTTP/1.1 ##这里为相对url，因此下面要有host域 来指定主机名
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: en.wikipedia.org
> Accept: */*
> #####-----------------------------------------------------以上为http request---------------------------------------------------------
< HTTP/1.0 200 OK
< Date: Tue, 29 Nov 2011 19:09:25 GMT
< Server: Apache
< X-Content-Type-Options: nosniff
< Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
< Content-Language: en
< Vary: Accept-Encoding,Cookie
< Last-Modified: Sat, 26 Nov 2011 21:36:43 GMT
< Content-Length: 41192
< Content-Type: text/html; charset=UTF-8
< Age: 131597
< X-Cache: HIT from sq76.wikimedia.org
< X-Cache-Lookup: HIT from sq76.wikimedia.org:3128
< X-Cache: MISS from sq72.wikimedia.org
< X-Cache-Lookup: MISS from sq72.wikimedia.org:80
< Connection: close ######----------------------------------------------以上为http response-------------------------------------------------------------
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 41192 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html lang="en" dir="ltr" class="client-nojs" xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>cURL - Wikipedia, the free encyclopedia</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="generator" content="MediaWiki 1.18wmf1" />
<link rel="alternate" type="application/x-wiki" title="Edit this page" href="/w/index.php?title=CURL&action=edit" />
<link rel="edit" title="Edit this page" href="/w/index.php?title=CURL&action=edit" />
<link rel="apple-touch-icon" href="//en.wikipedia.org/apple-touch-icon.png" />
<link rel="shortcut icon" href="/favicon.ico" />
<link rel="search" type="application/opensearchdescription+xml" href="/w/opensearch_desc.php" title="Wikipedia (en)" />
<link rel="EditURI" type="application/rsd+xml" href="//en.wikipedia.org/w/api.php?action=rsd" />
<link rel="copyright" href="//creativecommons.org/licenses/by-sa/3.0/" />
<link rel="alternate" type="application/atom+xml" title="Wikipedia Atom feed" href="/w/index.php?title=Special:RecentChanges&feed=atom" />
<link rel="stylesheet" href="//bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.wikihiero%7Cmediawiki.legacy.commonPrint%2Cshared%7Cskins.vector&only=styles&skin=vector&*" type="text/css" media="all" />
<meta name="ResourceLoaderDynamicStyles" content="" />
curl命令选项完整参考
[root@fsc01 scratch]# curl --help
Usage: curl [options...] <url>
Options: (H) means HTTP/HTTPS only, (F) means FTP only
-a/--append Append to target file when uploading (F)
-A/--user-agent <string> User-Agent to send to server (H)
--anyauth Pick "any" authentication method (H)
-b/--cookie <name=string/file> Cookie string or file to read cookies from (H)
--basic Use HTTP Basic Authentication (H)
-B/--use-ascii Use ASCII/text transfer
-c/--cookie-jar <file> Write cookies to this file after operation (H)
-C/--continue-at <offset> Resumed transfer offset
-d/--data <data> HTTP POST data (H) #########---------似乎可以用来提交表单 （post方法）
--data-ascii <data> HTTP POST ASCII data (H)
--data-binary <data> HTTP POST binary data (H)
--negotiate Use HTTP Negotiate Authentication (H)
--digest Use HTTP Digest Authentication (H)
--disable-eprt Inhibit using EPRT or LPRT (F)
--disable-epsv Inhibit using EPSV (F)
-D/--dump-header <file> Write the headers to this file
--egd-file <file> EGD socket path for random data (SSL)
--tcp-nodelay Use the TCP_NODELAY option
-e/--referer Referer URL (H)
-E/--cert <cert[:passwd]> Client certificate file and password (SSL)
--cert-type <type> Certificate file type (DER/PEM/ENG) (SSL)
--key <key> Private key file name (SSL)
--key-type <type> Private key file type (DER/PEM/ENG) (SSL)
--pass <pass> Pass phrase for the private key (SSL)
--engine <eng> Crypto engine to use (SSL). "--engine list" for list
--cacert <file> CA certificate to verify peer against (SSL)
--capath <directory> CA directory (made using c_rehash) to verify
peer against (SSL)
--ciphers <list> SSL ciphers to use (SSL)
--compressed Request compressed response (using deflate or gzip)
--connect-timeout <seconds> Maximum time allowed for connection
--create-dirs Create necessary local directory hierarchy
--crlf Convert LF to CRLF in upload
-f/--fail Fail silently (no output at all) on HTTP errors (H)
--ftp-account <data> Account data to send when requested by server (F)
--ftp-alternative-to-user <cmd> String to replace "USER [name]" (F)
--ftp-create-dirs Create the remote dirs if not present (F)
--ftp-method [multicwd/nocwd/singlecwd] Control CWD usage (F)
--ftp-pasv Use PASV/EPSV instead of PORT (F)
--ftp-skip-pasv-ip Skip the IP address for PASV (F)
--ftp-ssl Try SSL/TLS for the ftp transfer (F)
--ftp-ssl-reqd Require SSL/TLS for the ftp transfer (F)
-F/--form <name=content> Specify HTTP multipart POST data (H)
--form-string <name=string> Specify HTTP multipart POST data (H)
-g/--globoff Disable URL sequences and ranges using {} and []
-G/--get Send the -d data with a HTTP GET (H)
-h/--help This help text
-H/--header <line> Custom header to pass to server (H)
--ignore-content-length Ignore the HTTP Content-Length header
-i/--include Include protocol headers in the output (H/F)
-I/--head Show document info only
-j/--junk-session-cookies Ignore session cookies read from file (H)
--interface <interface> Specify network interface/address to use
--krb4 <level> Enable krb4 with specified security level (F)
-k/--insecure Allow connections to SSL sites without certs (H)
-K/--config Specify which config file to read
-l/--list-only List only names of an FTP directory (F)
--limit-rate <rate> Limit transfer speed to this rate
--local-port <num>[-num] Force use of these local port numbers
-L/--location Follow Location: hints (H)
--location-trusted Follow Location: and send authentication even
to other hostnames (H)
-m/--max-time <seconds> Maximum time allowed for the transfer
--max-redirs <num> Maximum number of redirects allowed (H)
--max-filesize <bytes> Maximum file size to download (H/F)
-M/--manual Display the full manual
-n/--netrc Must read .netrc for user name and password
--netrc-optional Use either .netrc or URL; overrides -n
--ntlm Use HTTP NTLM authentication (H)
-N/--no-buffer Disable buffering of the output stream
-o/--output <file> Write output to <file> instead of stdout
-O/--remote-name Write output to a file named as the remote file
-p/--proxytunnel Operate through a HTTP proxy tunnel (using CONNECT)
--proxy-anyauth Pick "any" proxy authentication method (H)
--proxy-basic Use Basic authentication on the proxy (H)
--proxy-digest Use Digest authentication on the proxy (H)
--proxy-ntlm Use NTLM authentication on the proxy (H)
-P/--ftp-port <address> Use PORT with address instead of PASV (F)
-q If used as the first parameter disables .curlrc
-Q/--quote <cmd> Send command(s) to server before file transfer (F)
-r/--range <range> Retrieve a byte range from a HTTP/1.1 or FTP server
--random-file <file> File for reading random data from (SSL)
-R/--remote-time Set the remote file's time on the local output
--retry <num> Retry request <num> times if transient problems occur
--retry-delay <seconds> When retrying, wait this many seconds between each
--retry-max-time <seconds> Retry only within this period
-s/--silent Silent mode. Don't output anything
-S/--show-error Show error. With -s, make curl show errors when they occur
--socks4 <host[:port]> Use SOCKS4 proxy on given host + port
--socks5 <host[:port]> Use SOCKS5 proxy on given host + port
--stderr <file> Where to redirect stderr. - means stdout
-t/--telnet-option <OPT=val> Set telnet option
--trace <file> Write a debug trace to the given file
--trace-ascii <file> Like --trace but without the hex output
--trace-time Add time stamps to trace/verbose output
-T/--upload-file <file> Transfer <file> to remote site
--url <URL> Spet URL to work with
-u/--user <user[:password]> Set server user and password
-U/--proxy-user <user[:password]> Set proxy user and password
-v/--verbose Make the operation more talkative
-V/--version Show version number and quit
-w/--write-out [format] What to output after completion
-x/--proxy <host[:port]> Use HTTP proxy on given port
-X/--request <command> Specify request command to use
-y/--speed-time Time needed to trig speed-limit abort. Defaults to 30
-Y/--speed-limit Stop transfer if below speed-limit for 'speed-time' secs
-z/--time-cond <time> Transfer based on a time condition
-0/--http1.0 Use HTTP 1.0 (H)
-1/--tlsv1 Use TLSv1 (SSL)
-2/--sslv2 Use SSLv2 (SSL)
-3/--sslv3 Use SSLv3 (SSL)
--3p-quote like -Q for the source URL for 3rd party transfer (F)
--3p-url source URL to activate 3rd party transfer (F)
--3p-user user and password for source 3rd party transfer (F)
-4/--ipv4 Resolve name to IPv4 address
-6/--ipv6 Resolve name to IPv6 address
-#/--progress-bar Display transfer progress as a progress bar