Twisted 网络编程介绍
前言：每个框架都是一座大厦。由于工作，我需要在短时间内熟悉Twisted。本系列博客不求深入洞悉Twisted内部各个部件，只求熟悉大厦的外观，和大厦提供的各种便利设施。
一 什么是Twisted ？
Twisted是一个用python写的事件驱动的网络编程框架。
支持一个堆网络传输协议 TCP, UDP, SSL/TLS,HTTP, XMPP, NNTP, IMAP, SSH, IRC, FTP, 好像我能想到的都支持。有12年的历史了，现在的版本是14.0.2
翻译一下维基百科上的介绍
Separation of protocols and transports
Twisted is designed for complete separation between logical protocols (usually relying on stream-based connection semantics, such as HTTP or POP3) and physical transport layers supporting such stream-based semantics (such as files, sockets or SSL libraries). Connection between a logical protocol and a transport layer happens at the last possible moment ― just before information is passed into the logical protocol instance. The logical protocol is informed of the transport layer instance, and can use it to send messages back and to check for the peer's identity. Note that it is still possible, in protocol code, to deeply query the transport layer on transport issues (such as checking a client-side SSL certificate). Naturally, such protocol code will fail (raise an exception) if the transport layer does not support such semantics.
实现了逻辑协议层和物理传输层的完全分离。 （底层的东西，不深究）
Deferreds
Central to the Twisted application model is the concept of a deferred (elsewhere called a future). A deferred is an instance of a class designed to receive and process a result which has not been computed yet, for example because it is based on data from a remote peer. Deferreds can be passed around, just like regular objects, but cannot be asked for their value. Each deferred supports a callback chain. When the deferred gets the value, it is passed to the functions on the callback chain, with the result of each callback becoming the input for the next. Deferreds make it possible to arrange to operate on the result of a function call before its value has become available. For example, if a deferred returns a string from a remote peer containing an IP address in quad format, a callback can be attached to translate it into a 32-bit number. Any user of the deferred can now treat it as a deferred returning a 32-bit number. This, and the related ability to define "errbacks" (callbacks which are called as error handlers), allows code to specify in advance what to do when an asynchronous event occurs, without stopping to wait for the event. In non-event-driven systems, for example using threads, the operating system incurs premature and additional overhead organizing threads each time a blocking call is made.
延时是一个类的对象。这个类用来接收和处理还未计算出来的结果。（说白了就是回调，就是支持非阻塞的IO操作。这个东西早已不新鲜,node.js大肆使用，看来这个wiki太老了，这样一个寻常的技术还被作为特性列出来）
Thread support
Twisted supports an abstraction over raw threads ― using a thread as a deferred source. Thus, a deferred is returned immediately, which will receive a value when the thread finishes. Callbacks can be attached which will run in the main thread, thus alleviating the need for complex locking solutions. A prime example of such usage, which comes from Twisted's support libraries, is using this model to call into databases. The database call itself happens on a foreign thread, but the analysis of the result happens in the main thread.
线程支持。使用一个线程作为延迟的源。一个延迟对象会被立即返回。减轻了发生复杂的锁的情况。（不足为奇的东西）
Foreign loop support
Twisted can integrate with foreign event loops, such as those of GTK+, Qt and Cocoa (through PyObjC). This allows using Twisted as the networking support layer in graphical user interface (GUI) programs, using all of its libraries without adding a thread-per-socket overhead, as using Python's native library would. A full-fledged web server can be integrated in-process with a GUI program using this model, for example.
可以与外部事件循环结合。可以作为GUI编程的网络支持层。一个功能齐全的Web服务器可以包含在一个GUI程序中。（不太理解）
二 安装Twisted
与很多library一样，只支持python2.x版本。windows平台直接官网下载msi安装文件来安装。
三 源码一瞥
twisted\web\http.py
# -*- test-case-name: twisted.web.test.test_http -*-
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
HyperText Transfer Protocol implementation.
This is the basic server-side protocol implementation used by the Twisted
Web server. It can parse HTTP 1.0 requests and supports many HTTP 1.1
features as well. Additionally, some functionality implemented here is
also useful for HTTP clients (such as the chunked encoding parser).
@var CACHED: A marker value to be returned from cache-related request methods to
indicate to the caller that a cached response will be usable and no response
body should be generated.
@var NOT_MODIFIED: An HTTP response code indicating that a requested
pre-condition (for example, the condition represented by an
I{If-Modified-Since} header is present in the request) has succeeded. This
indicates a response body cached by the client can be used.
@var PRECONDITION_FAILED: An HTTP response code indicating that a requested
pre-condition (for example, the condition represented by an I{If-None-Match}
header is present in the request) has failed. This should typically
indicate that the server has not taken the requested action.
"""
from __future__ import division, absolute_import
__all__ = [
'SWITCHING', 'OK', 'CREATED', 'ACCEPTED', 'NON_AUTHORITATIVE_INFORMATION',
'NO_CONTENT', 'RESET_CONTENT', 'PARTIAL_CONTENT', 'MULTI_STATUS',
'MULTIPLE_CHOICE', 'MOVED_PERMANENTLY', 'FOUND', 'SEE_OTHER',
'NOT_MODIFIED', 'USE_PROXY', 'TEMPORARY_REDIRECT',
'BAD_REQUEST', 'UNAUTHORIZED', 'PAYMENT_REQUIRED', 'FORBIDDEN', 'NOT_FOUND',
'NOT_ALLOWED', 'NOT_ACCEPTABLE', 'PROXY_AUTH_REQUIRED', 'REQUEST_TIMEOUT',
'CONFLICT', 'GONE', 'LENGTH_REQUIRED', 'PRECONDITION_FAILED',
'REQUEST_ENTITY_TOO_LARGE', 'REQUEST_URI_TOO_LONG',
'UNSUPPORTED_MEDIA_TYPE', 'REQUESTED_RANGE_NOT_SATISFIABLE',
'EXPECTATION_FAILED',
'INTERNAL_SERVER_ERROR', 'NOT_IMPLEMENTED', 'BAD_GATEWAY',
'SERVICE_UNAVAILABLE', 'GATEWAY_TIMEOUT', 'HTTP_VERSION_NOT_SUPPORTED',
'INSUFFICIENT_STORAGE_SPACE', 'NOT_EXTENDED',
'RESPONSES', 'CACHED',
'urlparse', 'parse_qs', 'datetimeToString', 'datetimeToLogString', 'timegm',
'stringToDatetime', 'toChunk', 'fromChunk', 'parseContentRange',
'StringTransport', 'HTTPClient', 'NO_BODY_CODES', 'Request',
'PotentialDataLoss', 'HTTPChannel', 'HTTPFactory',
]
# system imports
import tempfile
import base64, binascii
import cgi
import socket
import math
import time
import calendar
import warnings
import os
from io import BytesIO as StringIO
try:
from urlparse import (
ParseResult as ParseResultBytes, urlparse as _urlparse)
from urllib import unquote
from cgi import parse_header as _parseHeader
except ImportError:
from urllib.parse import (
ParseResultBytes, urlparse as _urlparse, unquote as _unquote)
from io import TextIOWrapper
def unquote(string, *args, **kwargs):
return _unquote(string.decode('charmap'), *args, **kwargs).encode('charmap')
def _parseHeader(line):
key, pdict = cgi.parse_header(line.decode('charmap'))
return (key.encode('charmap'), pdict)
from zope.interface import implementer, provider
# twisted imports
from twisted.python.compat import (
_PY3, unicode, intToBytes, networkString, nativeString)
from twisted.python import log
from twisted.python.components import proxyForInterface
from twisted.internet import interfaces, reactor, protocol, address
from twisted.internet.defer import Deferred
from twisted.protocols import policies, basic
from twisted.web.iweb import IRequest, IAccessLogFormatter
from twisted.web.http_headers import _DictHeaders, Headers
from twisted.web._responses import (
SWITCHING,
OK, CREATED, ACCEPTED, NON_AUTHORITATIVE_INFORMATION, NO_CONTENT,
RESET_CONTENT, PARTIAL_CONTENT, MULTI_STATUS,
MULTIPLE_CHOICE, MOVED_PERMANENTLY, FOUND, SEE_OTHER, NOT_MODIFIED,
USE_PROXY, TEMPORARY_REDIRECT,
BAD_REQUEST, UNAUTHORIZED, PAYMENT_REQUIRED, FORBIDDEN, NOT_FOUND,
NOT_ALLOWED, NOT_ACCEPTABLE, PROXY_AUTH_REQUIRED, REQUEST_TIMEOUT,
CONFLICT, GONE, LENGTH_REQUIRED, PRECONDITION_FAILED,
REQUEST_ENTITY_TOO_LARGE, REQUEST_URI_TOO_LONG, UNSUPPORTED_MEDIA_TYPE,
REQUESTED_RANGE_NOT_SATISFIABLE, EXPECTATION_FAILED,
INTERNAL_SERVER_ERROR, NOT_IMPLEMENTED, BAD_GATEWAY, SERVICE_UNAVAILABLE,
GATEWAY_TIMEOUT, HTTP_VERSION_NOT_SUPPORTED, INSUFFICIENT_STORAGE_SPACE,
NOT_EXTENDED,
RESPONSES)
if _PY3:
_intTypes = int
else:
_intTypes = (int, long)
protocol_version = "HTTP/1.1"
CACHED = """Magic constant returned by http.Request methods to set cache
validation headers when the request is conditional and the value fails
the condition."""
# backwards compatability
responses = RESPONSES
未完，可见，twisted用python实现了http协议。可以推测，很多协议都是用python本身实现的。
再看ftp.py这个文件比较短
# -*- test-case-name: twisted.test.test_ftp_options -*-
# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.
"""
I am the support module for making a ftp server with twistd.
"""
from twisted.application import internet
from twisted.cred import portal, checkers, strcred
from twisted.protocols import ftp
from twisted.python import usage, deprecate, versions
import warnings
class Options(usage.Options, strcred.AuthOptionMixin):
synopsis = """[options].
WARNING: This FTP server is probably INSECURE do not use it.
"""
optParameters = [
["port", "p", "2121", "set the port number"],
["root", "r", "/usr/local/ftp", "define the root of the ftp-site."],
["userAnonymous", "", "anonymous", "Name of the anonymous user."]
]
compData = usage.Completions(
optActions={"root": usage.CompleteDirs(descr="root of the ftp site")}
)
longdesc = ''
def __init__(self, *a, **kw):
usage.Options.__init__(self, *a, **kw)
self.addChecker(checkers.AllowAnonymousAccess())
def opt_password_file(self, filename):
"""
Specify a file containing username:password login info for
authenticated connections. (DEPRECATED; see --help-auth instead)
"""
self['password-file'] = filename
msg = deprecate.getDeprecationWarningString(
self.opt_password_file, versions.Version('Twisted', 11, 1, 0))
warnings.warn(msg, category=DeprecationWarning, stacklevel=2)
self.addChecker(checkers.FilePasswordDB(filename, cache=True))
def makeService(config):
f = ftp.FTPFactory()
r = ftp.FTPRealm(config['root'])
p = portal.Portal(r, config.get('credCheckers', []))
f.tld = config['root']
f.userAnonymous = config['userAnonymous']
f.portal = p
f.protocol = ftp.FTP
try:
portno = int(config['port'])
except KeyError:
portno = 2121
return internet.TCPServer(portno, f)
I am the support module for making a ftp server with twistd 这么搞笑的注释，哈哈
内置文档
The Vision For Twisted
Many other documents in this repository are dedicated to defining what Twisted is. Here, I will attempt to explain not what Twisted is, but what it should be, once I’ve met my goals with it.
First, Twisted should be fun. It began as a game, it is being used commercially in games, and it will be, I hope, an interactive and entertaining experience for the end-user.
Twisted is a platform for developing internet applications. While Python by itself is a very powerful language, there are many facilities it lacks which other languages have spent great attention to adding. It can do this now; Twisted is a good (if somewhat idiosyncratic) pure-python framework or library, depending on how you treat it, and it continues to improve.
As a platform, Twisted should be focused on integration. Ideally, all functionality will be accessible through all protocols. Failing that, all functionality should be configurable through at least one protocol, with a seamless and consistent user-interface. The next phase of development will be focusing strongly on a configuration system which will unify many disparate pieces of the current infrastructure, and allow them to be tacked together by a non-programmer.
Twisted的愿景
应该是好玩的。它始于游戏，正在用于游戏
一个纯python写的网络应用程序的平台
目前的发展，专注于整合已有的功能