20.1 介绍


20.1.1 Web应用:客户端/服务器计算


20.1.2 因特网


20.2 使用Python进行Web应用:创建一个简单的Web客户端


20.2.1 统一资源定位符


20.2.2 urlparse模块


urlparse(urlstr, defProtSch=None, allowFrag=None)


urlunparse(urltup)


urlparse.urljoin()


urljoin(baseurl, newurl,allowFrag=None)


20.2.3 urllib模块


urllib.urlopen()

urlopen(urlstr, postQueryData=None)


urllib.urlretrieve()


urlretrieve(urlstr, localfile=None, downloadStatusHook=None)


urllib.quote() and urllib,quote_plus()


urllib函数描述


urlopen(urlstr,postQurey-Data=None)


20.2.4 urllib2 模块


我们可以建立一个基础认证处理器(urllib2.HTTPBasicAuthHandler),同时在基


本URL或域上注册一个登陆密码,这就意味着我们在Web站点上定义了个安全区域,


一旦完成这些,你可以安装URL打开器,通过这个处理器打开所有的URL


另一个可选办法就是当浏览器提示的时候,输入用户名和密码,这样就发送了一个


带有适当用户请求的认证头


# vi urlopenAuth.py(问题)

-----------------------------------------

#!/usr/bin/env python


import urllib2


LOGIN = 'wesley'

PASSWD = "you'llNeverGuess"

URL = 'http://localhost'


def handler_version(url):

   from urlparse import urlparse as up

   hdlr = urllib2.HTTPBasicAuthHandler()

   hdlr.add_password('Archives', up(url)[1], LOGIN, PASSWD)

   opener = urllib2.build_opener(hdlr)

   urllib2.install_opener(opener)

   return url


def request_version(url):

   from base64 import encodestring

   req = urllib2.Request(url)

   b64str = encodestring('%s:%s' % (LOGIN, PASSWD))[:-1]

   req.add_header("Authorization", "Basic %s" % b64str)

   return req


for funcType in ('handler', 'request'):

   print '*** Using %s:' % funcType.upper()

   url = eval('%s_version')(URL)

   f = urllib2.urlopen(url)

   print f.readline()

   f.close()


-----------------------------------------


20.3 高级Web客户端


Web浏览器是基本的Web客户端,主要用来在Web上查询或者下载文件

高级Web客户端的一个例子就是网络爬虫,这些程序可以基于不同目的在因特网上


探索和下载页面:


1.为Google和Yahho这类大型搜索引擎建立索引

2.脱机浏览一将文档下载到本地,重新设定超链接,为本地浏览器创建镜像

3.下载并保存历史记录或框架

4.Web页的缓存,节省再次访问Web站点的下载时间


20.4 CGI:帮助Web服务器处理客户端数据


20.4.1 CGI介绍


CGI代表了在一个web服务器和能够处理用户表单,生成并返回动态HTML页的应用


程序间的交互


20.4.2 CGI应用程序

当一个CGI脚本开始执行时,它需要检索用户-支持表单,但这些数据必须要从web


客户端才可以获得,而不是从服务器或者硬盘上获得,所有的交互都将发生在Web


客户端,Web服务器端和CGI应用程序间


20.4.2 cgi模块


20.5 建立CGI应用程序


20.5.1 建立Web服务器


为了可以用Python进行CGI开发,首先需要安装一个Web服务器,将其配置成可以处


理Python CGI请求的模式,然后让你的Web服务器访问CGI脚本


1.可以下载安装apache及其Python CGI插件模块

2.利用Python自带的web服务器


# python -m CGIHTTPServer

---------------------------------

Serving HTTP on 0.0.0.0 port 8000 ...

---------------------------------


20.5.2 建立表单页


20.5.3 生成结果页


1.登陆一个非root账户,在当前目录下建立一个端口号为8000的web服务器

# cd ~

# pwd

---------------------

/home/python

---------------------

# mkdir web

# cd web

# python -m CGIHTTPServer


2.在启动这个服务器的目录下建立一个cgi-bin,并将Python CGI脚本放到该目录

# mkdir cgi-bin

# cd cgi-bin

# vi friends1.py

-------------------------

#!/usr/bin/env python


import cgi


reshtml = '''Content-Type: text/html\n

</p> <p>Friends CGI Demo (dynamic screen)</p> <p>

Friends list for: %s

Your name is: %s

You have %s friends.

'''


form = cgi.FieldStorage()

who = form['person'].value

howmany = form['howmany'].value

print reshtml % (who, who, howmany)

-------------------------


3.创建web表单:

这个HTML文件展示给用户一个空文档,含有用户名,和一系列可供用户选择的单选


按钮

# cd ..

# pwd

--------------------

/home/python/web

--------------------

# vi friends.htm

------------------------------------

CGI Demo(static screen)

Friends list for: NEW USER

Enter your Name:

How many friends do you have?

0

10

25

50

100

------------------------------------


表单的变量是FieldStorage的实例,包含person和howmany 字段值,我们把值存入


Python的who和howmany变量,变量reshtml包含需要返回的HTML文本正文,还有一


些动态填好的字段,这些数据都是从表单中读入的


4.浏览器访问页面

http://localhost:8000/friends.htm


20.5.4 生成表单和结果页面


将friends.html和friends1.py合并成friends2.py,得到的脚本可以同时显示表


单和动态生成的HTML结果页面,同时可以巧妙的知道应该输出哪个页面


# vi friends2.py

---------------------------------

#!/usr/bin/env python

'''

$Id: friends2.py,v 1.1 2000/12/31 01:32:45 wesc Exp $


CGI demo

'''


import cgi


header = 'Content-Type: text/html\n\n'


formhtml = '''Friends CGI Demo

Friends list for: NEW USER

Enter your Name:

How many friends do you have?

%s

'''


friendradio = ' %s\n'


def showForm():

   friends = ''

   for i in [0, 10, 25, 50, 100]:

       checked = ''

       if i == 0:

           checked = 'CHECKED'

       friends = friends + friendradio % (str(i), checked, str(i))

   print header + formhtml % (friends)



reshtml = '''Friends CGI Demo

Friends list for: %s

Your name is: %s

You have %s friends.

'''


def doResults(who, howmany):

   # substitute in real name and number of friends and return

   print header + reshtml % (who, who, howmany)



# process() does all the work

def process():


   # initialize Data class object

   form = cgi.FieldStorage()


   # get user name

   if form.has_key('person'):

       who = form['person'].value

   else:

       who = 'NEW USER'


   # get name and number of friends

   if form.has_key('howmany'):

       howmany = form['howmany'].value

   else:

       howmany = 0


   # if editing, show results

   if form.has_key('action'):

       doResults(who, howmany)


   # otherwise, show form

   else:

       showForm()


# invoke if called directly

if __name__ == '__main__':

   process()


---------------------------------


20.5.5 全面交互的web站点


我们最后一个例子将会完成这个循环

用户在表单页中输入他/她的信息,然后我们处理这些数据,并输出一个结果页面

现在我们将会在结果页面上加个链接允许返回到表单页面,但是我们返回的是含


有用户输入信息的页面而不是一个空白页面,我们页面上加上了一些错误处理程


序,来展示它是如何实现的


例,通过加上返回输入信息的表单页面连接,我们实现了整个循环,并加上一些错


误验证,在用户没有选择任何单选按钮时,通知用户


# vi friends3.py

--------------------------------

#!/usr/bin/env python


'''

$Id: friends3.py,v 1.1 2000/12/31 01:32:45 wesc Exp $


Friends CGI demo

'''


import cgi

from urllib import quote_plus

from string import capwords

#from sys import stderr

#s = stderr.write


header = 'Content-Type: text/html\n\n'

url = 'http://192.168.102.88:8000/cgi-bin/friends3.py'


errhtml = '''Friends CGI Demo

ERROR

%s

'''


# showError() --> None

def showError(error_str):

   'showError() -- display error message'

   print header + errhtml % (error_str)


friendradio = ' %s\n'


formhtml = '''Friends CGI Demo

Friends list for: %s

Your Name:

How many friends do you have?

%s

'''


# showForm() --> None

def showForm(who, howmany):

   'showForm() -- presents blank or data-filled form for new input'


   friends = ''

   for i in [0, 10, 25, 50, 100]:

       checked = ''

       if str(i) == howmany:

           checked = 'CHECKED'

       friends = friends + friendradio % (str(i), checked, str(i))

   print header + formhtml % (who, url, who, friends)


reshtml = '''Friends CGI Demo

Friends list for: %s

Your name is: %s

You have %s friends.

Click here to edit your data again.

'''


# doResults() --> None

def doResults(who, howmany):

   'doResults() -- displays results with given form data'


   # substitute in real name and number of friends and return

   newurl = url + '?action=reedit&person=%s&howmany=%s' %


(quote_plus(who), howmany)

   print header + reshtml % (who, who, howmany, newurl)


# process() --> None

def process():

   'process() does all the work:  grabs user data and determines


routine to call'


   error = ''


   # initialize Data class object

   form = cgi.FieldStorage()

   #s('name: '+str(form.name)+'\n')

   #s('keys: '+str(form.keys())+'\n')

   #for i in form.keys():

           #s('item: '+str(form[i].name)+' has a value of '+str(form


[i].value)+' and is a ' + form[i].__class__.__name__ + '\n')


   # get user name

   if form.has_key('person'):

       who = capwords(form['person'].value)

   else:

       who = 'NEW USER'


   # get name and number of friends

   if form.has_key('howmany'):

       howmany = form['howmany'].value

   else:

       if form.has_key('action') and form['action'].value == 'edit':

           error = 'Please select the number of friends you have.'

       else:

           howmany = 0


   # no errors, either display form or present results

   if not error:


       # if editing the first time, show results

       if form.has_key('action') and form['action'].value !=


'reedit':

           doResults(who, howmany)


       # otherwise, show form

       else:

           showForm(who, howmany)


   # send error message back if error situation

   else:

       showError(error)



# invoke if called directly

if __name__ == '__main__':

   process()

--------------------------------


20.6 在CGI中使用Unide编码


例,简单Unicode CGI示例(uniCGI.py)

这个脚本输出到你web浏览器端的是Unicode字符串


# vi uniCGI.py

--------------------------------

#!/usr/bin/env python


CODEC = 'UTF-8'

UNICODE_HELLO = u'''

Hello!

\u00A1Hola!

\u4F60\u597D!

\u3053\u3093\u306B\u3061\u306F!

'''


print 'Content-Type: text/html; charset=%s\r' % CODEC

print '\r'

print 'Unicode CGI Demo'

print ''

print UNICODE_HELLO.encode(CODEC)

print ''

--------------------------------


20.7 高级CGI


20.7.1 Mulitipart 表单提交和文件上传


20.7.2 多值字段


20.7.3 cookie


20.7.4 使用高级CGI


例,这个脚本有一个处理所有事情的主函数,AdvCGI,它有方法显示表单,错误或结


果页面,同事也可以从客户端(Web浏览器)读写cookie


# vi advcgi.py(问题)

----------------------------

#!/usr/bin/env python


from cgi import FieldStorage

from os import environ

from cStringIO import StringIO

from urllib import quote, unquote

from string import capwords, strip, split, join


class AdvCGI:


   header = 'Content-Type: text/html\n\n'

   url = '/py/advcgi.py'


   formhtml = '''</p> <p>Advanced CGI Demo

Advanced CGI Demo Form

My Cookie Setting

  • CPPuser = %s

    Enter cookie value

    (optional)

    Enter your name

    (required)

    What languages can you program in?

    (at least one required)

    %s

    Enter file to upload

    • '''


       langSet = ('Python', 'PERL', 'Java', 'C++', 'PHP',

                       'C', 'JavaScript')

       langItem = \

           ' %s\n'


       def getCPPCookies(self):                # reads cookies from


    client

           if environ.has_key('HTTP_COOKIE'):

               for eachCookie in map(strip, \

                       split(environ['HTTP_COOKIE'], ';')):

                   if len(eachCookie) > 6 and \

                           eachCookie[:3] == 'CPP':

                       tag = eachCookie[3:7]

                       try:

                           self.cookies[tag] = \

                               eval(unquote(eachCookie[8:]))

                       except (NameError, SyntaxError):

                           self.cookies[tag] = \

                               unquote(eachCookie[8:])

           else:

               self.cookies['info'] = self.cookies['user'] = ''


           if self.cookies['info'] != '':

               self.who, langStr, self.fn = \

                   split(self.cookies['info'], ':')

               self.langs = split(langStr, ',')

           else:

               self.who = self.fn = ''

               self.langs = ['Python']


       def showForm(self):                        # show fill-out form

           self.getCPPCookies()

           langStr = ''

           for eachLang in AdvCGI.langSet:

               if eachLang in self.langs:

                   langStr = langStr + AdvCGI.langItem % \

                       (eachLang, ' CHECKED', eachLang)

               else:

                   langStr = langStr + AdvCGI.langItem % \

                       (eachLang, '', eachLang)


           if not self.cookies.has_key('user') or \

                   self.cookies['user'] == '':

               cookieStatus = '(cookie has not been set yet)'

               userCook = ''

           else:

               userCook = cookieStatus = self.cookies['user']


           print AdvCGI.header + AdvCGI.formhtml % (AdvCGI.url,

               cookieStatus, userCook, self.who, langStr, self.fn)


       errhtml = '''</p> <p>Advanced CGI Demo

    ERROR

    %s

    ONCLICK="window.history.back()">

    '''


       def showError(self):

           print AdvCGI.header + AdvCGI.errhtml % (self.error)


       reshtml = '''</p> <p>Advanced CGI Demo

    Your Uploaded Data

    Your cookie value is: %s

    Your name is: %s

    You can program in the following languages:

      %s

    Your uploaded file...

    Name: %s

    Contents:

    %s

    Click here to return to form.

    '''


       def setCPPCookies(self):

           for eachCookie in self.cookies.keys():

               print 'Set-Cookie: CPP%s=%s; path=/' % \

                   (eachCookie, quote(self.cookies[eachCookie]))


       def doResults(self):

           MAXBYTES = 1024

           langlist = ''

           for eachLang in self.langs:

               langlist = langlist + '

  • %s
    ' % eachLang


           filedata = ''

           while len(filedata) < MAXBYTES:        # read file chunks

               data = self.fp.readline()

               if data == '': break

               filedata = filedata + data

           else:                                # truncate if too long

               filedata = filedata + \

                   '... (file truncated due to size)'

           self.fp.close()

           if filedata == '':

               filedata = \

                   '(file upload error or file not given)'

           filename = self.fn


           if not self.cookies.has_key('user') or \

                   self.cookies['user'] == '':

               cookieStatus = '(cookie has not been set yet)'

               userCook = ''

           else:

               userCook = cookieStatus = self.cookies['user']


           self.cookies['info'] = join([self.who, \

               join(self.langs, ','), filename], ':')

           self.setCPPCookies()

           print AdvCGI.header + AdvCGI.reshtml % \

               (cookieStatus, self.who, langlist,

               filename, filedata, AdvCGI.url)


       def go(self):                # determine which page to return

           self.cookies = {}

           self.error = ''

           form = FieldStorage()

           if form.keys() == []:

               self.showForm()

               return


           if form.has_key('person'):

               self.who = capwords(strip(form['person'].value))

               if self.who == '':

                   self.error = 'Your name is required. (blank)'

           else:

               self.error = 'Your name is required. (missing)'


           if form.has_key('cookie'):

               self.cookies['user'] = unquote(strip( \

                   form['cookie'].value))

           else:

               self.cookies['user'] = ''


           self.langs = []

           if form.has_key('lang'):

               langdata = form['lang']

               if type(langdata) == type([]):

                   for eachLang in langdata:

                       self.langs.append(eachLang.value)

               else:

                   self.langs.append(langdata.value)

           else:

               self.error = 'At least one language required.'


           if form.has_key('upfile'):

               upfile = form["upfile"]

               self.fn = upfile.filename or ''

               if upfile.file:

                   self.fp = upfile.file

               else:

                   self.fp = StringIO('(no data)')

           else:

               self.fp = StringIO('(no file)')

               self.fn = ''


           if not self.error:

               self.doResults()

           else:

               self.showError()


    if __name__ == '__main__':

       page = AdvCGI()

       page.go()


    ----------------------------


    20.8 Web(HTTP)服务器


    20.8.1 用Python建立Web服务器


    例,这个简单的Web服务器可以读取GET请求,获取Web页面并将其返回给客户端,它


    通过使用BaseHTTPServer的BaseHTTPRequestHandler处理器执行do_GET()方法来


    处理GET请求


    # vi myhttpd.py

    ----------------------------------

    #!/usr/bin/env python


    from os import curdir, sep

    from BaseHTTPServer import \

       BaseHTTPRequestHandler, HTTPServer


    class MyHandler(BaseHTTPRequestHandler):


       def do_GET(self):

           try:

               f = open(curdir + sep + self.path)

               self.send_response(200)

               self.send_header('Content-type',

    'text/html')

               self.end_headers()

               self.wfile.write(f.read())

               f.close()

           except IOError:

               self.send_error(404,

    'File Not Found: %s' % self.path)


    def main():

       try:

           server = HTTPServer(('', 80), MyHandler)

           print 'Welcome to the machine...'

    print 'Press ^C once or twice to quit'

           server.serve_forever()

       except KeyboardInterrupt:

           print '^C received, shutting down server'

           server.socket.close()


    if __name__ == '__main__':

       main()

    ----------------------------------


    20.9 相关模块

    ........