BT Protocol 规范1.0

 

BitTorrent 协议规范1.0版

2007-11-15 11:25

摘自http://wiki.theory.org /BitTorrentSpecification,标题为Bittorrent Protocol Specification v1.0。它比http://www.bitconjurer.org/BitTorrent/protocol.html介绍的详细多了。 Identification BitTorrent is a peer-to-peer file sharing protocol designed by Bram Cohen. Visit his pages at http://www.bitconjurer.org. BitTorrent is designed to facilitate file transfers among multiple peers across unreliable networks. Purpose The purpose of this specification is to document version 1.0 of the BitTorrent protocol specification in detail. Bram's protocol specification page http://www.bitconjurer.org/BitTorrent/protocol.html outlines the protocol in somewhat general terms, and lacks behaviorial detail in some areas. The hope is that this document will become a formal specification, written in clear, unambiguous terms, which can be used as a basis for discussion and implementation in the future. This document is intended to be maintained and used by the BitTorrent development community. Everyone is invited to contribute to this document, with the understanding that the content here is intended to represent the current protocol, which is already deployed in a number of client implementations. This is not the place to suggest feature requests. For that, please go to the mailing list. Scope This document applies to the first version (i.e. version 1.0) of the BitTorrent protocol specification. Currently, this applies to the torrent file structure, peer wire protocol, and the Tracker HTTP/HTTPS protocol specifications. As newer revisions of each protocol are defined, they should be specified on their own separate pages, not here. Related Documents http://www.bitconjurer.org/BitTorrent/protocol.html - The official protocol specification. BitTorrentWishList - A wish list for developers and end users alike. BitTorrentTrackerExtensions - Describes the various extensions of the Tracker protocol that are in use. Conventions In this document, a number of conventions are used in an attempt to present information in a concise and unambiguous fashion. peer v/s client: In this document, a peer is any BitTorrent client participating in a download. The client is also a peer, however it is the BitTorrent client that is running on the local machine. Reader of this specification may choose to think of themselves as the client which connects to numerous peers. piece v/s block: In this document, a piece refers to a portion of the downloaded data that is described in the metainfo file, which can be verified by a SHA1 hash. A block is a portion of data that a client may request from a peer. Two or more blocks make up a whole piece, which may then be verified. defacto standard: Large blocks of text in italics indicates a practice so common in various client implementations of BitTorrent that it is considered a defacto standard. In order help others to find recent changes that have been made to this document, please fill out the change log (last section). This should contain a brief (i.e. one-line) entry for each major change that you've made to the document. bencoding Bencoding is a way to specify and organize data in a terse format. It supports the following types: byte strings, integers, lists, and dictionaries. byte strings Byte strings are encoded as follows: <string length encoded in base ten ASCII>:<string data> Note that there is no constant beginning delimiter, and no ending delimiter. Example: 4:spam represents the string "spam" integers Integers are encoded as follows: i<integer encoded in base ten ASCII>e The initial i and trailing e are beginning and ending delimiters. You can have negative numbers such as i-3e. You cannot prefix the number with a zero such as i04e. However, i0e is valid. Example i3e represents the integer "3" lists Lists are encoded as follows: l<bencoded values>e The initial l and trailing e are beginning and ending delimiters. Lists may contain any bencoded type, including integers, strings, dictionaries, and other lists. Example: l4:spam4:eggse represents the list of two strings: ["spam", "eggs"] dictionaries Dictionaries are encoded as follows: d<bencoded string><bencoded element>e The initial d and trailing e are the beginning and ending delimiters. Note that the keys must be bencoded strings. The values may be any bencoded type, including integers, strings, lists, and other dictionaries. Keys must be strings and appear in sorted order (sorted as raw strings, not alphanumerics). Example: d3:cow3:moo4:spam4:eggse represents the dictionary { "cow" => "moo", "spam" => "eggs" } Example: d4:spaml1:a1:bee represents the dictionary { "spam" => ["a", "b"] } Metainfo File Structure All data in a metainfo file is bencoded. The specification for bencoding is defined above. The content of a metainfo file (the file ending in ".torrent") is a bencoded dictionary, containing the keys listed below. All character string values are UTF-8 encoded. Keys not marked 'optional' are required fields: info: a dictionary that describes the file(s) of the torrent. There are two possible forms: one for the case of a 'single-file' torrent with no directory structure, and one for the case of a 'multi-file' torrent, which can contain subdirectory trees. For the case of the single-file mode, the info dictionary contains the following structure length: length of the file in bytes (integer) md5sum: (optional) a 32-character hexadecimal string corresponding to the MD5 sum of the file. This is not used by BitTorrent at all, but it is included by some programs for greater compatibility. name: the filename of the file. This is purely advisory. (string) piece length: number of bytes in each piece (integer) pieces: string consisting of the concatenation of all 20-byte SHA1 hash values, one per piece (byte string) For the case of the multi-file mode, the info dictionary contains the following structure files: a list of dictionaries, one for each file. Each dictionary in this list contains the following keys: length: length of the file in bytes (integer) md5sum: (optional) a 32-character hexadecimal string corresponding to the MD5 sum of the file. This is not used by BitTorrent at all, but it is included by some programs for greater compatibility. path: a list containing one or more string elements that together represent the path and filename. Each element in the list corresponds to either a directory name or (in the case of the final element) the filename. For example, a the file "dir1/dir2/file.ext" would consist of three string elements: "dir1", "dir2", and "file.ext". This is encoded as a bencoded list of strings such as l4:dir14:dir28:file.exte name: the name of the top-most directory in the structure -- the directory which contains all of the files listed in the above files list. (character string) piece length: number of bytes in each piece (integer) pieces: string consisting of the concatenation of all 20-byte SHA1 hash values, one per piece (byte string) announce: The announce URL of the tracker (string) announce-list: (optional) this is an extention to the official specification, which is also backwards compatible. This key is used to implement lists of backup trackers. The full specification can be found at http://home.elp.rr.com/tur/multitracker-spec.txt creation date: (optional) the creation time of the torrent, in standard Unix epoch format (integer seconds since 1-Jan-1970 00:00:00 UTC) comment: (optional) free-form textual comments of the author (string) created by: (optional) name and version of the program used to create the .torrent (string) Notes The piece length specifies the nominal piece size, and is usually a power of 2. The piece size is typically chosen based on the total amount of file data in the torrent, constrained by the fact that too small a piece size will result in a large .torrent metadata file, and piece sizes too large cause inefficiency. The general rule of thumb seems to be to pick the smallest piece size that results in a .torrent file no greater than approx. 50 - 75 kB. The most common sizes are 256 kB, 512 kB, and 1 MB. Every piece is of equal length except for the final piece, which is irregular. The number of pieces is thus determined by 'ceil( total length / piece size )'. For the purposes of piece boundaries in the multi-file case, consider the file data as one long continuous stream, composed of the concatenation of each file in the order listed in the files list. The number of pieces and their boundaries are then determined in the same manner as the case of a single file. Pieces may overlap file boundaries. Each piece has a corresponding SHA1 hash of the data contained within that piece. These hashes are concatenated to form the pieces value in the above info dictionary. Note that this is not a list but rather a single string. The length of the string must be a multiple of 20 bytes. Tracker HTTP/HTTPS Protocol The tracker is an HTTP/HTTPS service which responds to HTTP GET requests. The requests include metrics from clients that help the tracker keep overall statistics about the torrent. The response includes a peer list that helps the client participate in the torrent. The base URL consists of the "announce URL" as defined in the metadata (.torrent) file. The parameters are then added to this URL, using standard CGI methods (i.e. a '?' after the announce URL, followed by 'param=value' sequences separated by '&') Note that all binary data in the URL (particularly info_hash and peer_id) must be properly escaped. This means any byte not in the set 0-9, a-z, A-Z, and $-_.+!*'(), must be encoded using the "%nn" format, where nn is the hexadecimal value of the byte. (See RFC1738 for details.) The parameters used in the client->tracker GET request are as follows: info_hash: 20-byte SHA1 hash of the value of the info key from the Metainfo file. Note that the value will be a bencoded dictionary, given the definition of the info key above. Note: This string is always urlencoded, as opposed to peer_id, which needs to be unencoded. peer_id: 20-byte string used as a unique ID for the client, generated by the client at startup. This is allowed to be any value, and may be binary data. There are currently no guidelines for generating this peer ID. However, one may rightly presume that it must at least be unique for your local machine, thus should probably incorporate things like process ID and perhaps a timestamp recorded at startup. See peer_id below for common client encodings of this field. port: The port number that the client is listening on. Ports reserved for BitTorrent are typically 6881-6889. Clients may choose to give up if it cannot establish a port within this range. uploaded: The total amount uploaded (since the client sent the 'started' event to the tracker) in base ten ASCII. While not explicitly stated in the official specification, the concensus is that this should be the total number of bytes uploaded. downloaded: The total amount downloaded (since the client sent the 'started' event to the tracker) in base ten ASCII. While not explicitly stated in the official specification, the consensus is that this should be the total number of bytes downloaded. left: The number of bytes this client still has to download, encoded in base ten ASCII. no_peer_id: Seems to be used by Experimental BitTorrent client and Clients such as ABC to tell the tracker if the client has no peer_id. 1 Means that the client has a peer id and 0 means unknown Peer id. Trackers should not allow this to be 0. compact: Used in some bittorrent trackers to check if the Tracker is in compact mode. event: If specified, must be one of started, completed, stopped, (or empty which is the same as not being specified). If not specified, then this request is one performed at regular intervals. started: The first request to the tracker must include the event key with the started value. stopped: Must be sent to the tracker if the client is shutting down gracefully. completed: Must be sent to the tracker when the download completes. However, must not be sent if the download was already 100% complete when the client started. Presumably, this is to allow the tracker to increment the "completed downloads" metric based soley on this event. ip: Optional. The true IP address of the client machine, in dotted quad format or rfc3513 defined hexed IPv6 address. Notes: In general this parameter is not necessary as the address of the client can be determined from the IP address from which the HTTP request came. The parameter is only needed in the case where the IP address that the request came in on is not the IP address of the client. This happens if the client is communicating to the tracker through a proxy (or a transparent web proxy/cache.) It also is necessary when both the client and the tracker are on the same local side of a NAT gateway. The reason for this is that otherwise the tracker would give out the internal (RFC1918) address of the client, which is not routeable. Therefore the client must explicitly state its (external, routeable) IP address to be given out to external peers. Various trackers treat this parameter differently. Some only honor it only if the IP address that the request came in on is in RFC1918 space. Others honor it unconditionally, while others ignore it completely. In case of IPv6 address (e.g.: 2001:db8:1:2::100) it indicates only that client can communicate via IPv6. numwant: Optional. Number of peers that the client would like to receive from the tracker. This value is permitted to be zero. If omitted, typically defaults to 50 peers. The tracker responds with "text/plain" document consisting of a bencoded dictionary with the following keys: failure reason: If present, then no other keys may be present. The value is a human-readable error message as to why the request failed (string). interval: Interval in seconds that the client should wait between sending regular requests to the tracker complete: number of peers with the entire file, i.e. seeders (integer) incomplete: number of non-seeder peers, aka "leechers" (integer) peers: The value is a list of dictionaries, each with the following keys: peer id: peer's self-selected ID, as described above for the tracker request (string) ip: peer's IP address (either IPv6 or IPv4) or DNS name (string) port: peer's port number (integer) As mentioned above, the list of peers is length 50 by default. If there are fewer peers in the torrent, then the list will be smaller. Otherwise, the tracker randomly selects peers to include in the response. The tracker may choose to implement a more intelligent mechanism for peer selection when responding to a request. For instance, reporting seeds to other seeders could be avoided. Clients may send a request to the tracker more often than the specified interval, if an event occurs (i.e. stopped or completed) or if the client needs to learn about more peers. However, it is considered bad practice to "hammer" on a tracker to get multiple peers. If a client wants a large peer list in the response, then it should specify the numwant parameter. Tracker 'scrape' Convention By convention most trackers support another form of request, which queries the state of a given torrent (or all torrents) that the tracker is managing. This is referred to as the "scrape page" because it automates the otherwise tedious process of "screen scraping" the tracker's stats page. The scrape URL is also a HTTP GET method, similar to the one described above. However the base URL is different. To derive the scrape URL use the following steps: Begin with the announce URL. Find the last '/' in it. If the text immediately following that '/' isn't 'announce' it will be taken as a sign that that tracker doesn't support the scrape convention. If it does, substitute 'scrape' for 'announce' to find the scrape page. Examples: (announce URL -> scrape URL) http://spam.com/announce -> http://spam.com/scrape http://spam.com/x/announce -> http://spam.com/x/scrape http://spam.com/announce.php -> http://spam.com/scrape.php http://spam.com/a -> (scrape not supported) http://spam.com/announce?x=2%0644 -> http://spam.com/scrape?x=2%0644 http://spam.com/announce?x=2/4 -> (scrape not supported) http://spam.com/x%064announce -> (scrape not supported) Note especially that entity unquoting is not to be done. This standard is documented by Bram in the BitTorrent development list archive: http://groups.yahoo.com/group/BitTorrent/message/3275 The scrape URL may be supplimented by the optional parameter info_hash, a 20-byte value as described above. This restricts the tracker's report to that particular torrent. Otherwise stats for all torrents that the tracker is managing are returned. Software authors are strongly encouraged to use the info_hash parameter when at all possible, to reduce the load and bandwidth of the tracker. The response of this HTTP GET method is a "text/plain" document consisting of a bencoded dictionary, containing the following keys files: a dictionary containing one key/value pair for each torrent for which there are stats. If info_hash was supplied and was valid, this dictionary will contain a single key/value. Each key consists of a 20-byte binary info_hash value. The value of that key is yet another nested dictionary containing the following: complete: number of peers with the entire file, i.e. seeders (integer) downloaded: total number of times the tracker has registered a completion ("event=complete", i.e. a client finished downloading the torrent) incomplete: number of non-seeder peers, aka "leechers" (integer) name: (optional) the torrent's internal name, as specified by the "name" file in the info section of the .torrent file Note that this response has three levels of dictionary nesting. Here's an example: d5:filesd20:....................d8:completei5e10:downloadedi50e10:incompletei10eeee Where .................... is the 20 byte info_hash and there are 5 seeders, 10 leechers, and 50