[PYTHON] 核心编程笔记(20.Web编程)
20.1 介绍
20.1.1 Web应用:客户端/服务器计算
20.1.2 因特网
20.2 使用Python进行Web应用:创建一个简单的Web客户端
20.2.1 统一资源定位符
20.2.2 urlparse模块
urlparse(urlstr, defProtSch=None, allowFrag=None)
urlunparse(urltup)
urlparse.urljoin()
urljoin(baseurl, newurl,allowFrag=None)
20.2.3 urllib模块
urllib.urlopen()
urlopen(urlstr, postQueryData=None)
urllib.urlretrieve()
urlretrieve(urlstr, localfile=None, downloadStatusHook=None)
urllib.quote() and urllib,quote_plus()
urllib函数描述
urlopen(urlstr,postQurey-Data=None)
20.2.4 urllib2 模块
我们可以建立一个基础认证处理器(urllib2.HTTPBasicAuthHandler),同时在基
本URL或域上注册一个登陆密码,这就意味着我们在Web站点上定义了个安全区域,
一旦完成这些,你可以安装URL打开器,通过这个处理器打开所有的URL
另一个可选办法就是当浏览器提示的时候,输入用户名和密码,这样就发送了一个
带有适当用户请求的认证头
# vi urlopenAuth.py(问题)
-----------------------------------------
#!/usr/bin/env python
import urllib2
LOGIN = 'wesley'
PASSWD = "you'llNeverGuess"
URL = 'http://localhost'
def handler_version(url):
from urlparse import urlparse as up
hdlr = urllib2.HTTPBasicAuthHandler()
hdlr.add_password('Archives', up(url)[1], LOGIN, PASSWD)
opener = urllib2.build_opener(hdlr)
urllib2.install_opener(opener)
return url
def request_version(url):
from base64 import encodestring
req = urllib2.Request(url)
b64str = encodestring('%s:%s' % (LOGIN, PASSWD))[:-1]
req.add_header("Authorization", "Basic %s" % b64str)
return req
for funcType in ('handler', 'request'):
print '*** Using %s:' % funcType.upper()
url = eval('%s_version')(URL)
f = urllib2.urlopen(url)
print f.readline()
f.close()
-----------------------------------------
20.3 高级Web客户端
Web浏览器是基本的Web客户端,主要用来在Web上查询或者下载文件
高级Web客户端的一个例子就是网络爬虫,这些程序可以基于不同目的在因特网上
探索和下载页面:
1.为Google和Yahho这类大型搜索引擎建立索引
2.脱机浏览一将文档下载到本地,重新设定超链接,为本地浏览器创建镜像
3.下载并保存历史记录或框架
4.Web页的缓存,节省再次访问Web站点的下载时间
20.4 CGI:帮助Web服务器处理客户端数据
20.4.1 CGI介绍
CGI代表了在一个web服务器和能够处理用户表单,生成并返回动态HTML页的应用
程序间的交互
20.4.2 CGI应用程序
当一个CGI脚本开始执行时,它需要检索用户-支持表单,但这些数据必须要从web
客户端才可以获得,而不是从服务器或者硬盘上获得,所有的交互都将发生在Web
客户端,Web服务器端和CGI应用程序间
20.4.2 cgi模块
20.5 建立CGI应用程序
20.5.1 建立Web服务器
为了可以用Python进行CGI开发,首先需要安装一个Web服务器,将其配置成可以处
理Python CGI请求的模式,然后让你的Web服务器访问CGI脚本
1.可以下载安装apache及其Python CGI插件模块
2.利用Python自带的web服务器
# python -m CGIHTTPServer
---------------------------------
Serving HTTP on 0.0.0.0 port 8000 ...
---------------------------------
20.5.2 建立表单页
20.5.3 生成结果页
1.登陆一个非root账户,在当前目录下建立一个端口号为8000的web服务器
# cd ~
# pwd
---------------------
/home/python
---------------------
# mkdir web
# cd web
# python -m CGIHTTPServer
2.在启动这个服务器的目录下建立一个cgi-bin,并将Python CGI脚本放到该目录
# mkdir cgi-bin
# cd cgi-bin
# vi friends1.py
-------------------------
#!/usr/bin/env python
import cgi
reshtml = '''Content-Type: text/html\n
<HTML><HEAD><TITLE>
Friends CGI Demo (dynamic screen)
</TITLE></HEAD>
<BODY><H3>Friends list for: <I>%s</I></H3>
Your name is: <B>%s</B><P>
You have <B>%s</B> friends.
</BODY></HTML>'''
form = cgi.FieldStorage()
who = form['person'].value
howmany = form['howmany'].value
print reshtml % (who, who, howmany)
-------------------------
3.创建web表单:
这个HTML文件展示给用户一个空文档,含有用户名,和一系列可供用户选择的单选
按钮
# cd ..
# pwd
--------------------
/home/python/web
--------------------
# vi friends.htm
------------------------------------
<HTML>
<HEAD>
<TITLE>CGI Demo(static screen)</TITLE>
</HEAD>
<BODY><H3>Friends list for: <I>NEW USER</I></H3>
<FORM ACTION='cgi-bin/friends1.py'>
<B>Enter your Name:</B>
<INPUT TYPE='text' NAME=person VALUE='NEW USER' SIZE=15>
<P><B>How many friends do you have?</B></P>
<INPUT TYPE='radio' NAME=howmany VALUE='0' CHECKED> 0
<INPUT TYPE='radio' NAME=howmany VALUE='10'> 10
<INPUT TYPE='radio' NAME=howmany VALUE='25'> 25
<INPUT TYPE='radio' NAME=howmany VALUE='50'> 50
<INPUT TYPE='radio' NAME=howmany VALUE='100'> 100
<P><INPUT TYPE=submit></P>
</FORM>
</BODY>
</HTML>
------------------------------------
表单的变量是FieldStorage的实例,包含person和howmany 字段值,我们把值存入
Python的who和howmany变量,变量reshtml包含需要返回的HTML文本正文,还有一
些动态填好的字段,这些数据都是从表单中读入的
4.浏览器访问页面
http://localhost:8000/friends.htm
20.5.4 生成表单和结果页面
将friends.html和friends1.py合并成friends2.py,得到的脚本可以同时显示表
单和动态生成的HTML结果页面,同时可以巧妙的知道应该输出哪个页面
# vi friends2.py
---------------------------------
#!/usr/bin/env python
'''
$Id: friends2.py,v 1.1 2000/12/31 01:32:45 wesc Exp $
CGI demo
'''
import cgi
header = 'Content-Type: text/html\n\n'
formhtml = '''<HTML><HEAD><TITLE>Friends CGI Demo</TITLE></HEAD>
<BODY><H3>Friends list for: <I>NEW USER</I></H3>
<FORM ACTION="/cgi-bin/friends2.py">
<B>Enter your Name:</B>
<INPUT TYPE=hidden NAME=action VALUE=edit>
<INPUT TYPE=text NAME=person VALUE="" SIZE=15>
<P><B>How many friends do you have?</B>
%s
<P><INPUT TYPE=submit></FORM></BODY></HTML>'''
friendradio = '<INPUT TYPE=radio NAME=howmany VALUE="%s" %s> %s\n'
def showForm():
friends = ''
for i in [0, 10, 25, 50, 100]:
checked = ''
if i == 0:
checked = 'CHECKED'
friends = friends + friendradio % (str(i), checked, str(i))
print header + formhtml % (friends)
reshtml = '''<HTML><HEAD><TITLE>Friends CGI Demo</TITLE></HEAD>
<BODY><H3>Friends list for: <I>%s</I></H3>
Your name is: <B>%s</B><P>
You have <B>%s</B> friends.
</BODY></HTML>'''
def doResults(who, howmany):
# substitute in real name and number of friends and return
print header + reshtml % (who, who, howmany)
# process() does all the work
def process():
# initialize Data class object
form = cgi.FieldStorage()
# get user name
if form.has_key('person'):
who = form['person'].value
else:
who = 'NEW USER'
# get name and number of friends
if form.has_key('howmany'):
howmany = form['howmany'].value
else:
howmany = 0
# if editing, show results
if form.has_key('action'):
doResults(who, howmany)
# otherwise, show form
else:
showForm()
# invoke if called directly
if __name__ == '__main__':
process()
---------------------------------
20.5.5 全面交互的web站点
我们最后一个例子将会完成这个循环
用户在表单页中输入他/她的信息,然后我们处理这些数据,并输出一个结果页面
现在我们将会在结果页面上加个链接允许返回到表单页面,但是我们返回的是含
有用户输入信息的页面而不是一个空白页面,我们页面上加上了一些错误处理程
序,来展示它是如何实现的
例,通过加上返回输入信息的表单页面连接,我们实现了整个循环,并加上一些错
误验证,在用户没有选择任何单选按钮时,通知用户
# vi friends3.py
--------------------------------
#!/usr/bin/env python
'''
$Id: friends3.py,v 1.1 2000/12/31 01:32:45 wesc Exp $
Friends CGI demo
'''
import cgi
from urllib import quote_plus
from string import capwords
#from sys import stderr
#s = stderr.write
header = 'Content-Type: text/html\n\n'
url = 'http://192.168.102.88:8000/cgi-bin/friends3.py'
errhtml = '''<HTML><HEAD><TITLE>Friends CGI Demo</TITLE></HEAD>
<BODY><H3>ERROR</H3>
<B>%s</B><P>
<FORM><INPUT TYPE=button VALUE=Back ONCLICK="window.history.back
()"></FORM>
</BODY></HTML>'''
# showError() --> None
def showError(error_str):
'showError() -- display error message'
print header + errhtml % (error_str)
friendradio = '<INPUT TYPE=radio NAME=howmany VALUE="%s" %s> %s\n'
formhtml = '''<HTML><HEAD><TITLE>Friends CGI Demo</TITLE></HEAD>
<BODY><H3>Friends list for: <I>%s</I></H3>
<FORM ACTION="%s">
<B>Your Name:</B>
<INPUT TYPE=hidden NAME=action VALUE=edit>
<INPUT TYPE=text NAME=person VALUE="%s" SIZE=15>
<P><B>How many friends do you have?</B>
%s
<P><INPUT TYPE=submit></FORM></body></html>'''
# showForm() --> None
def showForm(who, howmany):
'showForm() -- presents blank or data-filled form for new input'
friends = ''
for i in [0, 10, 25, 50, 100]:
checked = ''
if str(i) == howmany:
checked = 'CHECKED'
friends = friends + friendradio % (str(i), checked, str(i))
print header + formhtml % (who, url, who, friends)
reshtml = '''<HTML><HEAD><TITLE>Friends CGI Demo</TITLE></HEAD>
<BODY><H3>Friends list for: <I>%s</I></H3>
Your name is: <B>%s</B><P>
You have <B>%s</B> friends.
<P>Click <a href="%s">here</a> to edit your data again.
</BODY></HTML>'''
# doResults() --> None
def doResults(who, howmany):
'doResults() -- displays results with given form data'
# substitute in real name and number of friends and return
newurl = url + '?action=reedit&person=%s&howmany=%s' %
(quote_plus(who), howmany)
print header + reshtml % (who, who, howmany, newurl)
# process() --> None
def process():
'process() does all the work: grabs user data and determines
routine to call'
error = ''
# initialize Data class object
form = cgi.FieldStorage()
#s('name: '+str(form.name)+'\n')
#s('keys: '+str(form.keys())+'\n')
#for i in form.keys():
#s('item: '+str(form[i].name)+' has a value of '+str(form
[i].value)+' and is a ' + form[i].__class__.__name__ + '\n')
# get user name
if form.has_key('person'):
who = capwords(form['person'].value)
else:
who = 'NEW USER'
# get name and number of friends
if form.has_key('howmany'):
howmany = form['howmany'].value
else:
if form.has_key('action') and form['action'].value == 'edit':
error = 'Please select the number of friends you have.'
else:
howmany = 0
# no errors, either display form or present results
if not error:
# if editing the first time, show results
if form.has_key('action') and form['action'].value !=
'reedit':
doResults(who, howmany)
# otherwise, show form
else:
showForm(who, howmany)
# send error message back if error situation
else:
showError(error)
# invoke if called directly
if __name__ == '__main__':
process()
--------------------------------
20.6 在CGI中使用Unide编码
例,简单Unicode CGI示例(uniCGI.py)
这个脚本输出到你web浏览器端的是Unicode字符串
# vi uniCGI.py
--------------------------------
#!/usr/bin/env python
CODEC = 'UTF-8'
UNICODE_HELLO = u'''
Hello!
\u00A1Hola!
\u4F60\u597D!
\u3053\u3093\u306B\u3061\u306F!
'''
print 'Content-Type: text/html; charset=%s\r' % CODEC
print '\r'
print '<HTML><HEAD><TITLE>Unicode CGI Demo</TITLE></HEAD>'
print '<BODY>'
print UNICODE_HELLO.encode(CODEC)
print '</BODY></HTML>'
--------------------------------
20.7 高级CGI
20.7.1 Mulitipart 表单提交和文件上传
20.7.2 多值字段
20.7.3 cookie
20.7.4 使用高级CGI
例,这个脚本有一个处理所有事情的主函数,AdvCGI,它有方法显示表单,错误或结
果页面,同事也可以从客户端(Web浏览器)读写cookie
# vi advcgi.py(问题)
----------------------------
#!/usr/bin/env python
from cgi import FieldStorage
from os import environ
from cStringIO import StringIO
from urllib import quote, unquote
from string import capwords, strip, split, join
class AdvCGI:
header = 'Content-Type: text/html\n\n'
url = '/py/advcgi.py'
formhtml = '''<HTML><HEAD><TITLE>
Advanced CGI Demo</TITLE></HEAD>
<BODY><H2>Advanced CGI Demo Form</H2>
<FORM METHOD=post ACTION="%s" ENCTYPE="multipart/form-data">
<H3>My Cookie Setting</H3>
<LI> <CODE><B>CPPuser = %s</B></CODE>
<H3>Enter cookie value<BR>
<INPUT NAME=cookie value="%s"> (<I>optional</I>)</H3>
<H3>Enter your name<BR>
<INPUT NAME=person VALUE="%s"> (<I>required</I>)</H3>
<H3>What languages can you program in?
(<I>at least one required</I>)</H3>
%s
<H3>Enter file to upload</H3>
<INPUT TYPE=file NAME=upfile VALUE="%s" SIZE=45>
<P><INPUT TYPE=submit>
</FORM></BODY></HTML>'''
langSet = ('Python', 'PERL', 'Java', 'C++', 'PHP',
'C', 'JavaScript')
langItem = \
'<INPUT TYPE=checkbox NAME=lang VALUE="%s"%s> %s\n'
def getCPPCookies(self): # reads cookies from
client
if environ.has_key('HTTP_COOKIE'):
for eachCookie in map(strip, \
split(environ['HTTP_COOKIE'], ';')):
if len(eachCookie) > 6 and \
eachCookie[:3] == 'CPP':
tag = eachCookie[3:7]
try:
self.cookies[tag] = \
eval(unquote(eachCookie[8:]))
except (NameError, SyntaxError):
self.cookies[tag] = \
unquote(eachCookie[8:])
else:
self.cookies['info'] = self.cookies['user'] = ''
if self.cookies['info'] != '':
self.who, langStr, self.fn = \
split(self.cookies['info'], ':')
self.langs = split(langStr, ',')
else:
self.who = self.fn = ''
self.langs = ['Python']
def showForm(self): # show fill-out form
self.getCPPCookies()
langStr = ''
for eachLang in AdvCGI.langSet:
if eachLang in self.langs:
langStr = langStr + AdvCGI.langItem % \
(eachLang, ' CHECKED', eachLang)
else:
langStr = langStr + AdvCGI.langItem % \
(eachLang, '', eachLang)
if not self.cookies.has_key('user') or \
self.cookies['user'] == '':
cookieStatus = '<I>(cookie has not been set yet)</I>'
userCook = ''
else:
userCook = cookieStatus = self.cookies['user']
print AdvCGI.header + AdvCGI.formhtml % (AdvCGI.url,
cookieStatus, userCook, self.who, langStr, self.fn)
errhtml = '''<HTML><HEAD><TITLE>
Advanced CGI Demo</TITLE></HEAD>
<BODY><H3>ERROR</H3>
<B>%s</B><P>
<FORM><INPUT TYPE=button VALUE=Back
ONCLICK="window.history.back()"></FORM>
</BODY></HTML>'''
def showError(self):
print AdvCGI.header + AdvCGI.errhtml % (self.error)
reshtml = '''<HTML><HEAD><TITLE>
Advanced CGI Demo</TITLE></HEAD>
<BODY><H2>Your Uploaded Data</H2>
<H3>Your cookie value is: <B>%s</B></H3>
<H3>Your name is: <B>%s</B></H3>
<H3>You can program in the following languages:</H3>
<UL>%s</UL>
<H3>Your uploaded file...<BR>
Name: <I>%s</I><BR>
Contents:</H3>
<PRE>%s</PRE>
Click <A HREF="%s"><B>here</B></A> to return to form.
</BODY></HTML>'''
def setCPPCookies(self):
for eachCookie in self.cookies.keys():
print 'Set-Cookie: CPP%s=%s; path=/' % \
(eachCookie, quote(self.cookies[eachCookie]))
def doResults(self):
MAXBYTES = 1024
langlist = ''
for eachLang in self.langs:
langlist = langlist + '<LI>%s<BR>' % eachLang
filedata = ''
while len(filedata) < MAXBYTES: # read file chunks
data = self.fp.readline()
if data == '': break
filedata = filedata + data
else: # truncate if too long
filedata = filedata + \
'... <B><I>(file truncated due to size)</I></B>'
self.fp.close()
if filedata == '':
filedata = \
'<B><I>(file upload error or file not given)</I></B>'
filename = self.fn
if not self.cookies.has_key('user') or \
self.cookies['user'] == '':
cookieStatus = '<I>(cookie has not been set yet)</I>'
userCook = ''
else:
userCook = cookieStatus = self.cookies['user']
self.cookies['info'] = join([self.who, \
join(self.langs, ','), filename], ':')
self.setCPPCookies()
print AdvCGI.header + AdvCGI.reshtml % \
(cookieStatus, self.who, langlist,
filename, filedata, AdvCGI.url)
def go(self): # determine which page to return
self.cookies = {}
self.error = ''
form = FieldStorage()
if form.keys() == []:
self.showForm()
return
if form.has_key('person'):
self.who = capwords(strip(form['person'].value))
if self.who == '':
self.error = 'Your name is required. (blank)'
else:
self.error = 'Your name is required. (missing)'
if form.has_key('cookie'):
self.cookies['user'] = unquote(strip( \
form['cookie'].value))
else:
self.cookies['user'] = ''
self.langs = []
if form.has_key('lang'):
langdata = form['lang']
if type(langdata) == type([]):
for eachLang in langdata:
self.langs.append(eachLang.value)
else:
self.langs.append(langdata.value)
else:
self.error = 'At least one language required.'
if form.has_key('upfile'):
upfile = form["upfile"]
self.fn = upfile.filename or ''
if upfile.file:
self.fp = upfile.file
else:
self.fp = StringIO('(no data)')
else:
self.fp = StringIO('(no file)')
self.fn = ''
if not self.error:
self.doResults()
else:
self.showError()
if __name__ == '__main__':
page = AdvCGI()
page.go()
----------------------------
20.8 Web(HTTP)服务器
20.8.1 用Python建立Web服务器
例,这个简单的Web服务器可以读取GET请求,获取Web页面并将其返回给客户端,它
通过使用BaseHTTPServer的BaseHTTPRequestHandler处理器执行do_GET()方法来
处理GET请求
# vi myhttpd.py
----------------------------------
#!/usr/bin/env python
from os import curdir, sep
from BaseHTTPServer import \
BaseHTTPRequestHandler, HTTPServer
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
try:
f = open(curdir + sep + self.path)
self.send_response(200)
self.send_header('Content-type',
'text/html')
self.end_headers()
self.wfile.write(f.read())
f.close()
except IOError:
self.send_error(404,
'File Not Found: %s' % self.path)
def main():
try:
server = HTTPServer(('', 80), MyHandler)
print 'Welcome to the machine...'
print 'Press ^C once or twice to quit'
server.serve_forever()
except KeyboardInterrupt:
print '^C received, shutting down server'
server.socket.close()
if __name__ == '__main__':
main()
----------------------------------
20.9 相关模块
........