python打造流媒体服务器,Python：HTTP发布带有流媒体的大文件

I'm uploading potentially large files to a web server. Currently I'm doing this:

import urllib2

f = open('somelargefile.zip','rb')

request = urllib2.Request(url,f.read())

request.add_header("Content-Type", "application/zip")

response = urllib2.urlopen(request)

However, this reads the entire file's contents into memory before posting it. How can I have it stream the file to the server?

解决方案

Reading through the mailing list thread linked to by systempuntoout, I found a clue towards the solution.

The mmap module allows you to open file that acts like a string. Parts of the file are loaded into memory on demand.

Here's the code I'm using now:

import urllib2

import mmap

# Open the file as a memory mapped string. Looks like a string, but

# actually accesses the file behind the scenes.

f = open('somelargefile.zip','rb')

mmapped_file_as_string = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)

# Do the request

request = urllib2.Request(url, mmapped_file_as_string)

request.add_header("Content-Type", "application/zip")

response = urllib2.urlopen(request)

#close everything

mmapped_file_as_string.close()

f.close()