To find which peers are hosting the content you’re after (discovery), IPFS uses a distributed hash table, or DHT. A hash table is a database of keys to values. A distributed hash table is one where the table is split across all the peers in a distributed network. To find content, you ask these peers.
要找到哪个节点持有你想要的内容(这个过程叫做discovery
),IPFS使用分布式哈希表(DHT
)。哈希表是一个存储键值的数据库。分布式哈希表就是将表分割后分给分布式网络中的所有对等节点。要找到内容,你可以问这些对等节点。
Once you know where your content is (or more precisely, which peers are storing each of the blocks that make up the content you’re after), you use the DHT again to find the current location of those peers (routing). So, in order to get to content, you use libp2p to query the DHT twice.
一旦你知道了你的内容在哪里(或者更准确地说,哪些节点正在存储你想要的每个数据块),你就可以再次使用DHT来查找这些节点的当前位置(这个过程叫做routing
)。因此,为了获得内容,要使用libp2p查询DHT两次。
Distributed Hash Tables (DHTs) are distributed key-value stores where keys are cryptographic hashes.
DHTs are, by definition, distributed. Each “peer” (or “node”) is responsible for a subset of the DHT.
When a peer receives a request, it either answers it, or the request is passed to another peer until a peer that can answer it is found.
分布式散列表(DHTs)是分布式键值对存储,其中的键是加密哈希值。
根据定义,DHT是分布式的。每个节点负责DHT的一个子集。
当一个节点接收到请求时,它要么应答请求,要么将请求传递给另一个节点,直到找到可以应答请求的节点为止。
Depending on the implementation, a request not answered by the first node contacted can be:
forwarded from peer to peer, with the last peer contacting the requesting peer
forwarded from peer to peer, with the answer forwarded following the same path
answered with the contact information of a node that has better chances to be able to answer. IPFS uses this strategy.
根据协议的实现,如果一个请求没有被第一个连接的节点应答的话,就会:
DHTs’ decentralization provides advantages compared to a traditional key-value store, including:
scalability
, since a request for a hash of length n takes at most log2(n) steps to resolve.
fault tolerancevia redundancy
, so that lookups are possible even if peers unexpectedly leave or join the DHT. Additionally, requests can be addressed to any peer if another peer is slow or unavailable.
load balancing
, since requests are made to different nodes and no unique peers process all the requests.
与传统的键值存储相比,DHTs的去中心化的优势在于:
可伸缩性
,因为对长度为n的哈希的请求最多需要log2(n)个步骤来解决。通过冗余实现容错
,这样即使节点意外离开或加入DHT,也可以进行查找。此外,如果另一个节点速度慢或不可用,则可以向任何节点发送请求。负载平衡
,因为请求是向不同的节点发出的,并且没有唯一的节点处理所有请求。Each peer has a peerID, which is a hash with the same length n as the DHT keys.
每个节点都有ID,叫做peerID
,是一个与DHT中的键key
具有相同长度的哈希值。
A subset of the DHT maintained by a peer is called a ‘bucket’.
A bucket maps to hashes with the same prefix as the peerID, up to m bits. There are 2^m buckets. Each bucket maps for 2^(n-m) hashes.
每个节点维护的一小部分DHT叫做桶bucket
.
bucket映射的是哈希值,这些哈希值与peerID具有相同的(最多m个)前缀。整个DHT总共有2^m
个bucket,每个bucket能够映射2^(n-m)
个哈希值。
For example, if m=2^16 and we use hexadecimal encoding (four bits per displayed character), the peer with peerID ‘ABCDEF12345’ maintains mapping for hashes starting with ‘ABCD’.
Some hashes falling into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB, just as examples.
// `m=2^16这个地方我觉得官方可能是写错了,应该是m=16`
例如,如果m=2^16并且我们使用十六进制编码(每个显示字符四位),peerID为“ABCDEF12345”的节点将维护以“ABCD”开头的哈希的映射。
一些落入这个桶中的哈希值将是ABCD38E56、ABCD09CBA或ABCD17ABB。
The size of a bucket is related to the size of the prefix. The longer the prefix, the fewer hashes each peer has to manage, and the more peers are needed.
Several peers can be in charge of the same bucket if they have the same prefix.
In most DHTs, including IPFS’s Kademlia implementation, the size of the buckets (and the size of the prefix), are dynamic.
bucket的大小与前缀的大小有关。前缀越长,每个节点需要管理的哈希值就越少,需要的节点就越多。
如果多个节点具有相同的前缀,则它们可能负责相同的bucket。
在大多数DHT中,包括IPFS的Kademlia实现,bucket的大小(和前缀的大小)是动态的。
Peers also keep a connection to other peers in order to forward requests if the requested hash is not in their own bucket.
节点与其他一些节点保持着连接,以便转发那些哈希值不在自己桶中的请求。
If hashes are of length n, a peer will keep n-1 lists of peers:
- the first list contains peers whose IDs have a different first bit.
- the second list contains peers whose IDs have first bits identical to its own, but a different second bit
…- the m-th list contains peers whose IDs have their first m-1 bits identical, but a different m-th bit
…
如果哈希值长度为n,那么一个节点就有n-1个节点列表:
The higher m is, the harder it is to find peers that have the same ID up to m bits. The lists of “closest” peers typically remains empty.
“Close” here is defined as the XOR distance, so the longer the prefix they share, the closer they are.
Lists also have a maximum of entries (k) — otherwise the first lists would contain half the network, then a fourth of the network, and so on.
m越高,就越难找到具有相同ID(最多m位)的节点。“最近的”节点列表通常保持为空。
这里的“近的”定义为异或距离,因此它们共享的前缀越长,它们就越接近。
列表有条目限制,最多为k——否则第一个列表将包含一半网络的节点,然后是四分之一,依此类推。
When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or answer with the contacting information (IP+port, peerID, etc.) of a closer peer. The requesting peer can then send its request to this closer peer. The process goes on until a peer is able to answer it.
A request for a hash of length n will take at maximum log2(n) steps, or even log2m(n).
当一个节点接收到一个查找请求时,如果它在自己的bucket中,它就把对应的值返回,否则返回一个更接近的节点的连接信息(IP+端口、peerID等)。然后,请求节点可以将其请求发送到此较近的节点。这个过程一直持续到某个节点能够回答为止。
长度为n的哈希值请求最多需要log2(n)步,甚至只需要log2m(n)步。
In IPFS’s Kademlia DHT, keys are SHA256 hashes. PeerIDs are those of libp2p, the networking library used by IPFS.
We use the DHT to look up two types of objects, both represented by SHA256 hashes:
- Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content.
- IPNS records. A lookup will give the last Content ID associated with this IPNS address, enabling the routing of mutable content.
我们使用DHT查找两种类型的对象,它们都由SHA256哈希表示:
Consequently, IPFS’s DHT is one of the ways to achieve mutable and immutable content routing. It’s currently the only one implemented.
因此,IPFS的DHT是一种实现可变内容和不可变内容路由的方法,也是目前唯一实现的方法。
附上部分代码
type Routing interface {
ContentRouting
PeerRouting
ValueStore
// Kicks off the bootstrap process.
Bootstrap(context.Context) error
}
// ContentRouting is used to find information about who has what content.
type ContentRouting interface {
// Provide adds the given CID to the content routing system. If 'true' is
// passed, it also announces it, otherwise it is just kept in the local
// accounting of which objects are being provided.
Provide(context.Context, cid.Cid, bool) error
// Search for peers who are able to provide a given key.
FindProvidersAsync(context.Context, cid.Cid, int) <-chan pstore.PeerInfo
}
// PeerRouting is a way to find information about certain peers.
//
// This can be implemented by a simple lookup table, a tracking server,
// or even a DHT (like herein).
type PeerRouting interface {
// FindPeer searches for a peer with given ID, returns a pstore.PeerInfo
// with relevant addresses.
FindPeer(context.Context, peer.ID) (pstore.PeerInfo, error)
}
// ValueStore is a basic Put/Get interface.
type ValueStore interface {
// PutValue adds value corresponding to given Key.
PutValue(context.Context, string, []byte, ...ropts.Option) error
// GetValue searches for the value corresponding to given Key.
GetValue(context.Context, string, ...ropts.Option) ([]byte, error)
}