1.数据文件及介绍
1.1 Sqlite数据
| 文件 | 包含表 | 内容 |
|---|---|---|
| Ledger.db | Ledgers | 区块信息 |
| Ledger.db | Validations | 本地历史区块共识信息 |
| Transaction.db | AccountTransactions | 账户交易表 |
| Transaction.db | Transactions | 交易相关信息 |
| Wallet.db | NodeIdentity | 存储当前节点的NodePublic与NodePrivate |
| Wallet.db | PublisherManifests | 没什么用 |
| Wallet.db | ValidatorManifests | 没什么用 |
1.2 序列化数据的存储
- NuDB 可用在各个平台上
- RocksDB 不可用在Windows平台上
2.数据结构
2.1 Sqlite表结构及说明
2.1.1 Ledgers
| 列名 | 类型 | 含义 |
|---|---|---|
| LedgerHash | CHARACTER | 哈希值 |
| LedgerSeq | BIGINT UNSIGNED | Ledger序号 |
| PrevHash | CHARACTER | 前个Ledger的Hash值 |
| TotalCoins | BIGINT UNSIGNED | 当前网络上的XRP总数（交易会销毁XRP） |
| ClosingTime | BIGINT UNSIGNED | 关闭时间 |
| PrevClosingTime | BIGINT UNSIGNED | 前一个区块的关闭时间 |
| CloseTimeRes | BIGINT UNSIGNED | ledger关闭时间的解决方案（2-120S） |
| CloseFlags | BIGINT UNSIGNED | 标识这个ledger是怎么关闭的，一般都是0 |
| AccountSetHash | CHARACTER | stateMap根结点hash |
| TransSetHash | CHARACTER | txMap 根节点哈希 |
2.1.2 Validations
| 列名 | 类型 | 含义 |
|---|---|---|
| LedgerSeq | BIGINT UNSIGNED | Ledger序号 |
| InitialSeq | BIGINT UNSIGNED | 共LedgerSeq一样 |
| LedgerHash | CHARACTER | 共识过程中用到的LedgerHash |
| NodePubKey | CHARACTER | 对ledger签名的节点公钥 |
| SignTime | BIGINT UNSIGNED | 签名时间 |
| RawData | BLOB | 与ledgerInfo类似的数据 |
2.1.3 AccountTransactions
| 列名 | 类型 | 含义 |
|---|---|---|
| TransID | CHARACTER | 交易hash |
| Account | CHARACTER | 账户ID |
| LedgerSeq | BIGINT UNSIGNED | ledger序号 |
| TxnSeq | INTEGER | 交易Sequence号(是此账户的第几个交易) |
2.1.4 Transactions
| 列名 | 类型 | 含义 |
|---|---|---|
| TransID | CHARACTER | 交易hash |
| TransType | CHARACTER | 交易类型 |
| FromAcct | CHARACTER | 交易的发起账户 |
| FromSeq | BIGINT UNSIGNED | 交易在账户中的序号 |
| LedgerSeq | BIGINT UNSIGNED | 交易落在哪个区块上 |
| Status | CHARACTER | 交易的状态V表示“共识过” |
| RawTxn | BLOB | 交易序列化数据 |
| TxnMeta | BLOB | 交易metaData的序列化数据 |
2.1.5 NodeIdentity
| 列名 | 类型 | 含义 |
|---|---|---|
| PublicKey | CHARACTER | 当前节点的NodePublic |
| PrivateKey | CHARACTER | 储当前节点的NodePrivate |
2.1.6 PublisherManifests
| 列名 | 类型 | 含义 |
|---|---|---|
| RawData | BLOB |
2.1.7 ValidatorManifests
| 列名 | 类型 | 含义 |
|---|---|---|
| RawData | BLOB |
2.2 序列化数据
2.2.1 Ripple的数据序列化
以LedgerInfo为例：
void addRaw (LedgerInfo const& info, Serializer& s)
{
s.add32 (info.seq);
s.add64 (info.drops.drops ());
s.add256 (info.parentHash);
s.add256 (info.txHash);
s.add256 (info.accountHash);
s.add32 (info.parentCloseTime.time_since_epoch().count());
s.add32 (info.closeTime.time_since_epoch().count());
s.add8 (info.closeTimeResolution.count());
s.add8 (info.closeFlags);
}
int Serializer::add16 (std::uint16_t i)
{
int ret = mData.size ();
mData.push_back (static_cast (i >> 8));
mData.push_back (static_cast (i & 0xff));
return ret;
}
2.2.2 存到NuDB的数据及序列化
需要序列化的数据分三种类型
/** The types of node objects. */
enum NodeObjectType
{
hotUNKNOWN = 0,
hotLEDGER = 1,
//hotTRANSACTION = 2 // Not used
hotACCOUNT_NODE = 3,
hotTRANSACTION_NODE = 4
};
- LedgerInfo (对应hotLEDGER)
- LedgerSeq,LedgerHash等，具体可参见Ripple官网
- 区块之间是通过LedgerHash与PrevHash（上一个区块哈希）来产生顺序关联的
- LedgerInfo的序列化代码：
// Save the ledger header in the hashed object store { Serializer s (128); s.add32 (HashPrefix::ledgerMaster); addRaw(ledger->info(), s); app.getNodeStore ().store ( hotLEDGER, std::move (s.modData ()), ledger->info().hash); } - StateMap (包括各种SLE信息)
- SLE是STLedgerEntry的简写，是Ripple自定义的一种数据结构
- SLE种类:Account、Escrow、Fee、Amendment、PayChannel等
- 其中有一个比较重要的SLE是skipList，它包含前面256个区块的ledgerHash，这个SLE在无交易的情况下占用空间最大，每个区块hash占32字节，256个就是8k大小
- StateMap序列化代码：
void Ledger::rawInsert(std::shared_ptrconst& sle) { Serializer ss; sle->add(ss); auto item = std::make_shared< SHAMapItem const>(sle->key(), std::move(ss)); // VFALCO NOTE addGiveItem should take ownership if (! stateMap_->addGiveItem( std::move(item), false, false)) LogicError("Ledger::rawInsert: key already exists"); } - TxMap (交易信息)
- 包含交易信息txnData与交易的元数据metaData(包含交易影响的结构等)
- 最终txnData与metaDat合到一块存储，也叫metaData
- TxMap的序列化代码
void Ledger::rawTxInsert (uint256 const& key, std::shared_ptrconst& txn, std::shared_ptr< Serializer const> const& metaData) { assert (metaData); // low-level - just add to table Serializer s(txn->getDataLength () + metaData->getDataLength () + 16); s.addVL (txn->peekData ()); s.addVL (metaData->peekData ()); auto item = std::make_shared< SHAMapItem const> (key, std::move(s)); if (! txMap().addGiveItem (std::move(item), true, true)) LogicError("duplicate_tx: " + to_string(key)); }
说明:
- StateMap与TxMap都是SHAMap类型的结构
- SHAMap既是基数树(Radix Tree)同时也是默克尔树(Merkle Tree)
- SHAMap中包含 SHAMapTreeNode 与 SHAMapInnerNode 两种节点类型
- SHAMapTreeNode SHAMap的非叶子节点
- SHAMapInnerNode SHAMap的叶子节点
- StateMap中的叶子节点对应hotACCOUNT_NODE，txMap中的叶子节点对应hotTRANSACTION_NODE类型
- SHAMap分支构造及查找算法代码：
最开始都是从root_节点找起，根据hash选择分支：
// Which branch would contain the specified hash
int SHAMapNodeID::selectBranch (uint256 const& hash) const
{
int branch = * (hash.begin () + (mDepth / 2));
if (mDepth & 1)
branch &= 0xf;
else
branch >>= 4;
assert ((branch >= 0) && (branch < 16));
return branch;
}
如果对应分支没有节点，则直接插入
// easy case, we end on an inner node
auto inner = std::static_pointer_cast(node);
int branch = nodeID.selectBranch (tag);
assert (inner->isEmptyBranch (branch));
auto newNode = std::make_shared (item, type, seq_);
inner->setChild (branch, newNode);
否则，构造一个InnerNode,并连同原来的叶子节点一起作为新InnerNode的叶子节点
auto leaf = std::static_pointer_cast(node);
auto inner = std::make_shared(seq_);
inner->setChildren(leaf, std::make_shared(item, type, seq_));
assert(!stack.empty());
auto parent = unshareNode(
std::static_pointer_cast(stack.top().first),
stack.top().second);
stack.top().first = parent;
node = inner;
setChildren
void
SHAMapInnerNodeV2::setChildren(std::shared_ptr const& child1,
std::shared_ptr const& child2)
{
assert(child1->peekItem()->key() != child2->peekItem()->key());
auto k1 = child1->peekItem()->key().begin();
auto k2 = child2->peekItem()->key().begin();
auto k = common_.begin();
for (depth_ = 0; *k1 == *k2; ++depth_, ++k1, ++k2, ++k)
*k = *k1;
unsigned b1;
unsigned b2;
if ((*k1 & 0xF0) == (*k2 & 0xF0))
{
*k = *k1 & 0xF0;
b1 = *k1 & 0x0F;
b2 = *k2 & 0x0F;
depth_ = 2*depth_ + 1;
}
else
{
b1 = *k1 >> 4;
b2 = *k2 >> 4;
depth_ = 2*depth_;
}
mChildren[b1] = child1;
mIsBranch |= 1 << b1;
mChildren[b2] = child2;
mIsBranch |= 1 << b2;
}
序列化数据存储相关类图：
image.png
序列化相关调用堆栈：
LedgerConsensusImp::beginAccept ->
LedgerConsensusImp::accept(855) ->
OpenView::apply ->
Ledger::rawReplace/Ledger::rawInsert(序列化) ->
SHAMap::addGiveItem/SHAMap::updateGiveItem
2.3 序列化数据存储到NuDB过程
- NodeObject的hash(uint256)元素取前4个字节得到key
- NodeObject 的 data经过lz4压缩算法压缩得到dataSize与dataCompressed
- dataSize,key,dataCompressed写入到文件
3.从DB中查找交易、区块、账户信息
3.1 查找区块信息
根据区块序号查找获得区块信息，如 ledger_data 命令：
- 直接从Ledger表读取LedgerHash或者读取SkipList获取LedgerHash（只对当前最大区块前256个区块或256整数倍的区块有效）
- 读取Ledger表，查找LedgerHash对应的记录信息并构造LedgerInfo
- 使用LedgerInfo及从配置文件中读取的db,nudb配置构造Ledger对象
- Ledger构造函数中
- 根据LedgerInfo中的初始化txMap
- 根据accountHash初始化stateMap
- 初始化Map过程中需要读取NuDB文件获取到root_节点信息
- 区块读取完毕
{
"result": {
"ledger": {
"accepted": true,
"account_hash": "EA4088D9B6FF34DA6102E4F6BCEC96DF860CF8006E66B544EEAD784635887514",
"close_flags": 0,
"close_time": 561191060,
"close_time_human": "2017-Oct-13 06:24:20",
"close_time_resolution": 20,
"closed": true,
"hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784",
"ledger_hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784",
"ledger_index": "8",
"parent_close_time": 561191044,
"parent_hash": "C1EFB978755E1011376C4DEFA7B61BA3834C4F218B85DC4718F0C77A0CB446B2",
"seqNum": "8",
"totalCoins": "99999999999999990",
"total_coins": "99999999999999990",
"transaction_hash": "35098D46F21556B3496DC8409CD1F51AEA9568B6935D43736EB046278747769B"
},
"ledger_hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784",
"ledger_index": 8,
"state": [
{
"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q",
"Balance": "10000000000",
"Flags": 0,
"LedgerEntryType": "AccountRoot",
"OwnerCount": 0,
"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC",
"PreviousTxnLgrSeq": 8,
"Sequence": 1,
"index": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99"
},
{
"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"Balance": "99999989999999990",
"Flags": 0,
"LedgerEntryType": "AccountRoot",
"OwnerCount": 0,
"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC",
"PreviousTxnLgrSeq": 8,
"Sequence": 2,
"index": "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8"
},
{
"Flags": 0,
"Hashes": [
"AB868A6CFEEC779C2FF845C0AF00A642259986AF40C01976A7F842B6918936C7",
"EF422714FD6900B2F5D69CA543FC3C67091E4178F514894CCDF11E30570FA90C",
"07EAC861B72B4F29F4825F695A203BAAD414E4A02DDAE7DB5C5F8B0B8A5ADFAE",
"847EA2FBAAB54AE6D084AC6A42DF1582907E0BC11C52A4E7D36F5A6424E832BA",
"69C031109212CB336A6E5A0D8BAB03E7A185711C48376EAFAC6B0458261D0CAF",
"1D3B71E935B8922E7FAD2D3E18EC36FD6884E1DBC6DE965A2BCFC27057DCAF68",
"C1EFB978755E1011376C4DEFA7B61BA3834C4F218B85DC4718F0C77A0CB446B2"
],
"LastLedgerSequence": 7,
"LedgerEntryType": "LedgerHashes",
"index": "B4979A36CDC7F3D3D5C31A4EAE2AC7D7209DDA877588B9AFC66799692AB0D66B"
}
],
"status": "success",
"validated": true
}
}
注：
- transaction_hash 不为空说明此Ledger上存在交易
- state字段为此Ledger中包含的SLE信息
3.2 查找帐户信息
给出帐户地址，查找帐户信息，如account_info命令
- 查找参数列表中有无ledger_index参数，如果有，去获取对应的Ledger，参考3.1，如果没有，去获取最新共识过的Ledger
- 用1中获取到的Ledger去读取account参数对应的SLE节点信息，读取过程通过Ledger中的stateMap结构去查找SLE的key对应的叶子节点
- 根据Ledger信息与得到的SLE构造返回结果
{
"result": {
"account_data": {
"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q",
"Balance": "10000000000",
"Flags": 0,
"LedgerEntryType": "AccountRoot",
"OwnerCount": 0,
"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC",
"PreviousTxnLgrSeq": 8,
"Sequence": 1,
"index": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99"
},
"ledger_hash": "50EE9E422B7591BD0B7EE7126FAFF8437F785D137DEB60E3BD98294CBA1175D1",
"ledger_index": 14,
"status": "success",
"validated": true
}
}
3.3 查找交易信息
给出交易Hash，查找交易详情的过程,如tx命令
- 根据交易Hash，从Transaction表中读取交易信息得到Transaction对象，其中，RawTxn字段内容可以构造原始交易信息，LedgerSeq字段标识交易落在哪个区块上，Status字段标识交易的共识状态，V表示共识通过
- 根据Transaction表读取到的LedgerSeq去读取Ledger信息，参考3.1
- 根据读取到的Ledger信息去查对应txMap中Hash对应的Node，Node的中包含txnData与metaData
- 将1中得到的Transaction对象与3中得到的metaData构造json，返回json，结束
{
"result": {
"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"Amount": "10000000000",
"Destination": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q",
"Fee": "10",
"Flags": 2147483648,
"Sequence": 1,
"SigningPubKey": "0330E7FC9D56BB25D6893BA3F317AE5BCF33B3291BD63DB32654A313222F7FD020",
"TransactionType": "Payment",
"TxnSignature": "30440220249D1FA7DBB71A0EAE07AF84289136FB8F4B16B774E255A8E2B41F08EB7EEA5302201566B52AD408760EC940E8930A1402CCF0F3CCBCBDDAF356D9D399EDFA4F89B1",
"date": 561191060,
"hash": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC",
"inLedger": 8,
"ledger_index": 8,
"meta": {
"AffectedNodes": [
{
"CreatedNode": {
"LedgerEntryType": "AccountRoot",
"LedgerIndex": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99",
"NewFields": {
"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q",
"Balance": "10000000000",
"Sequence": 1
}
}
},
{
"ModifiedNode": {
"FinalFields": {
"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"Balance": "99999989999999990",
"Flags": 0,
"OwnerCount": 0,
"Sequence": 2
},
"LedgerEntryType": "AccountRoot",
"LedgerIndex": "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8",
"PreviousFields": {
"Balance": "100000000000000000",
"Sequence": 1
}
}
}
],
"TransactionIndex": 0,
"TransactionResult": "tesSUCCESS",
"delivered_amount": "10000000000"
},
"status": "success",
"validated": true
}
}
注：
其实Transaction表中也存有metaData字段，不知道这里为什么要读文件去取
3.4 查找帐户交易信息
查找一个帐户下的交易，可限定交易数量，最大一次查找200个交易
{
"method": "account_tx",
"params": [
{
"account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"ledger_index_max": 1,
"ledger_index_min": 1000,
"limit": 20
}
]
}
这个过程完全从SqliteDB中进行，未涉及NuDB
- 从AccountTransactions表查找交易
- 根据查询结果构造交易信息
- 每个交易添加 validated:true，条件是给出的ledger_index_min 与ledger_index_max范围是当前共识过区块范围的子集
SELECT AccountTransactions.LedgerSeq,AccountTransactions.TxnSeq,
Status,RawTxn,TxnMeta
FROM AccountTransactions INNER JOIN Transactions
ON Transactions.TransID = AccountTransactions.TransID
AND AccountTransactions.Account = '%s' WHEREAccountTransactions.LedgerSeq BETWEEN '%u' AND '%u'
ORDER BY AccountTransactions.LedgerSeq DESC,
AccountTransactions.TxnSeq DESC
LIMIT %u;
返回结果
{
"id" : 1,
"result" : {
"account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"ledger_index_max" : 958,
"ledger_index_min" : 1,
"status" : "success",
"transactions" : [
{
"meta" : {
"AffectedNodes" : [
{
"ModifiedNode" : {
"FinalFields" : {
"Account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"Balance" : "99999989999999990",
"Flags" : 0,
"OwnerCount" : 0,
"Sequence" : 2
},
"LedgerEntryType" : "AccountRoot",
"LedgerIndex" : "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8",
"PreviousFields" : {
"Balance" : "100000000000000000",
"Sequence" : 1
}
}
},
{
"CreatedNode" : {
"LedgerEntryType" : "AccountRoot",
"LedgerIndex" : "81843E2DE3A90BADB1CA75B3C3781CFC72BDFD1584CA893240F317A7003FC93F",
"NewFields" : {
"Account" : "rwqbtoxtmwEzCatTocFW8TcP3DYU18GtGg",
"Balance" : "10000000000",
"Sequence" : 1
}
}
}
],
"TransactionIndex" : 0,
"TransactionResult" : "tesSUCCESS",
"delivered_amount" : "10000000000"
},
"tx" : {
"Account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh",
"Amount" : "10000000000",
"Destination" : "rwqbtoxtmwEzCatTocFW8TcP3DYU18GtGg",
"Fee" : "10",
"Flags" : 2147483648,
"Sequence" : 1,
"SigningPubKey" : "0330E7FC9D56BB25D6893BA3F317AE5BCF33B3291BD63DB32654A313222F7FD020",
"TransactionType" : "Payment",
"TxnSignature" : "30450221008BC728BB5F58FE91E109BBEF1B3F098E17E49EF91EEF12CC7EB065483CF056700220450C9382096F3A2EEC8F054B47C48A91035985429C3F7594D9176D881B7AB7D4",
"date" : 560931910,
"hash" : "3395C1AC406DB2974B72C1811008F8959D953802B88CB22E5B0231D6B0F8CE54",
"inLedger" : 20,
"ledger_index" : 20
},
"validated" : true
}
]
}