关于如何爬取数字货币交易信息
数字货币的交易，表现形式就是数字货币在地址间的流动。主要分为两种类型，一种是有主链的数字货币的流动信息，一种是代币的流动信息。
有主链的数字货币流动信息可以通过挖掘区块信息，获取区块中每一笔交易的流向得出。以莱特币为例：莱特币的每一个区块中，有若干交易信息，每一笔交易信息中，包括了多条目的记录和源纪录，每一条目的记录有流向的目标地址和数字货币的数目，但是每一笔源记录可没有源地址和数字货币的数目。那么，问题来了，数字货币是从哪个地址流出的呢？每一笔源记录对应的地址和数字货币的数目是多少呢？那只好再用莱特币客户端按照源记录中记录的交易哈希在去查询一次了，这个查询结果是一笔交易，这笔交易有多个目的记录，怎么确定该笔源记录对应源交易中哪笔目的纪录呢，源记录中有字段（vout）表示对应的是源交易中的第几笔目的纪录。
这样，我们就能确认一次资金的流向了，但是，这个流向不一定是交易的。如果源纪录包含coinbase字段则是挖矿产生，这个就不是交易记录了。其他情况也有可能是热钱包冷钱包互相转账，这个需要通过分析地址获取具体的交易记录，因为，每个交易所的地址实际上是有规律的。
例子，一个莱特币block：
{
"hash": "5f3657ba49d63db21563e456453a7a84176f037ec9b33d9ec2eab6ad055d6f77",
"confirmations": 872,
"strippedsize": 215,
"size": 215,
"weight": 860,
"height": 233,
"version": 1,
"versionHex": "00000001",
"merkleroot": "7753b2a996a27da49e80dbe08e0e6ce4d95626da2bc7077f1032ecd0cf2ce9c0",
"tx": [
"7753b2a996a27da49e80dbe08e0e6ce4d95626da2bc7077f1032ecd0cf2ce9c0"
],
"time": 1318475591,
"mediantime": 1318475579,
"nonce": 4942,
"bits": "1e0ffff0",
"difficulty": 0.000244140625,
"chainwork": "000000000000000000000000000000000000000000000000000000000ea00ea0",
"previousblockhash": "b7553c1e8f7c29a745284f5e0272e87450e5047722fd4f93512de28391916417",
"nextblockhash": "7465863a829ec0367b6d93ba4955b8628f5cc982fb4d6e41ff512f34924a40d1"
}
莱特的一笔挖矿交易（vin代表源记录，vout代表目的记录）：
{
"txid": "7753b2a996a27da49e80dbe08e0e6ce4d95626da2bc7077f1032ecd0cf2ce9c0",
"hash": "7753b2a996a27da49e80dbe08e0e6ce4d95626da2bc7077f1032ecd0cf2ce9c0",
"size": 134,
"vsize": 134,
"version": 1,
"locktime": 0,
"vin": [
{
"coinbase": "044757964e0101",
"sequence": 4294967295
}
],
"vout": [
{
"value": 50.00000000,
"n": 0,
"scriptPubKey": {
"asm": "04509423d0bad7821efc05cd01b702ca479d871be8b4ec80caa6b8bdf4521e6adc5be4f6688f7818b5e04fed590c21dc2d9470d6f3c53931dff5bb1a953adda3b4 OP_CHECKSIG",
"hex": "4104509423d0bad7821efc05cd01b702ca479d871be8b4ec80caa6b8bdf4521e6adc5be4f6688f7818b5e04fed590c21dc2d9470d6f3c53931dff5bb1a953adda3b4ac",
"reqSigs": 1,
"type": "pubkey",
"addresses": [
"Lhuxi3MegDB6G5HJXn7iDXYCjjJeASksSK"
]
}
}
]
}
莱特币一笔转移（vin源记录有一笔，其中的txid和vout字段标识源记录的交易hash和在源记录交易中的第几笔目的记录，该笔源记录是源交易a68603a36295411e385d0e81e93a30971815b2c5c9f4ad804c12b31854737e3c中的第0笔目的纪录，vout目的纪录有两笔，value表示每一笔目的纪录的转移数目，addresses字段标识目的纪录的地址）：
{
"txid": "a25e496c5806c6ef1450176601f759fd28af404fd64228005d63a4e762fddc6e",
"hash": "a25e496c5806c6ef1450176601f759fd28af404fd64228005d63a4e762fddc6e",
"size": 258,
"vsize": 258,
"version": 1,
"locktime": 0,
"vin": [
{
"txid": "a68603a36295411e385d0e81e93a30971815b2c5c9f4ad804c12b31854737e3c",
"vout": 0,
"scriptSig": {
"asm": "3045022100911cc41db021b836343c6ca300cf260d551d609ee063c162eb3da6e801d4e542022075b7801722465fe99e472ecb76fa4d1152b90eb55dbc4c86dc23d2f1d346ce27[ALL] 0438e14c7e63456582fe669063a71adf3c2242e75548f57dc0e8a7ca11936fbb2fcbd76334ee67ab9f022ac2c841c06ecc4bb6ee844e4a6c5cff3e28327d02b533",
"hex": "483045022100911cc41db021b836343c6ca300cf260d551d609ee063c162eb3da6e801d4e542022075b7801722465fe99e472ecb76fa4d1152b90eb55dbc4c86dc23d2f1d346ce2701410438e14c7e63456582fe669063a71adf3c2242e75548f57dc0e8a7ca11936fbb2fcbd76334ee67ab9f022ac2c841c06ecc4bb6ee844e4a6c5cff3e28327d02b533"
},
"sequence": 4294967295
}
],
"vout": [
{
"value": 6.86484600,
"n": 0,
"scriptPubKey": {
"asm": "OP_DUP OP_HASH160 cc0eea1686e05f5457610ceb4ccdbea643882b4e OP_EQUALVERIFY OP_CHECKSIG",
"hex": "76a914cc0eea1686e05f5457610ceb4ccdbea643882b4e88ac",
"reqSigs": 1,
"type": "pubkeyhash",
"addresses": [
"LdpuycmdLcERxResHd5D9nxsas9vbNxVa7"
]
}
},
{
"value": 0.78826300,
"n": 1,
"scriptPubKey": {
"asm": "OP_DUP OP_HASH160 c1c6c0462c953f3d9779b31146685b71949b6fd3 OP_EQUALVERIFY OP_CHECKSIG",
"hex": "76a914c1c6c0462c953f3d9779b31146685b71949b6fd388ac",
"reqSigs": 1,
"type": "pubkeyhash",
"addresses": [
"LctYmhiQ1RwXjA8iyRFJBLQrSvvjSm5AdM"
]
}
}
]
}
代币以基于eth的代币做例子。eth的交易分为转账交易和合约交易，代币的转移发生在合约交易。eth每一笔交易都有一个目的地址，如果目的地址为空，是创建合约，该合约的合约地址在“contractAddress”字段记录。如果目的地址不为空，可能是正常的eth的转移，也可能是合约交易。如果是合约交易，则目的地址是合约地址。合约交易有多种操作，不一定是代币的转移。代币的转移主要通过“logs”字段分析得出，logs字段有多笔日志，每一笔日志记录信息主要在“topics”和“data”字段，“topics”字段的第一个值是操作的sha3签名，其中选取sha3('Transfer(address,address,uint256)')这个操作即可。这个操作就是代币的转移。后续参数可能在“topics”的后续值，也可能在“data”字段中。如果”topics”字段后续值数目不能满足Transfer函数参数数目，剩余的参数的值在“data”字段中存储。详见：https://codeburst.io/deep-dive-into-ethereum-logs-a8d2047c7371。这样，就可以通过分析日志获取代币资金的流向了， 当然要排除合约地址流出的代币，这些并不是交易产生。
例子：一个以太坊的block解析，粗体为交易（有from和to，to为合约地址则是合约交易）：
{"jsonrpc":"2.0","id":1,"result":{"difficulty":"0xc6d2fa46fd6","extraData":"0xd783010400844765746887676f312e352e31856c696e7578","gasLimit":"0x2fefd8","gasUsed":"0xa410","hash":"0x9a834c53bbee9c2665a5a84789a1d1ad73750b2d77b50de44f457f411
d02e52e","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000","miner":"0x68795c4aa09d6f4ed3e5deddf8c2ad3049a601da","mixHash":"0x72224fd0b7b32d0fea13f2953eba28d2512af43632725ed75583dd30e42dfc64","nonce":"0xf1a453a44f9de627","number":
"0x100000","parentHash":"0x2dcaf9a8a3fd329d925eb47c1f22022d4f2d562500e546d878bad4247d013858","receiptsRoot":"0x178000485de197ed06a4ebada431864e5b564b43f994fb8888521f74b4b3257b","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948
a7413f0a142fd40d49347","size":"0x303","stateRoot":"0xeaddaa0b6b673ba3f8cb7235ca656340ccf703295cf5252cb04a4202c47252c2","timestamp":"0x56cc7b73","totalDifficulty":"0x6bab93cdf810b22a","transactions":[{"blockHash":"0x9a834c53bbee9c2665a5a
84789a1d1ad73750b2d77b50de44f457f411d02e52e","blockNumber":"0x100000","from":"0x68795c4aa09d6f4ed3e5deddf8c2ad3049a601da","gas":"0x15f90","gasPrice":"0xba43b7400","hash":"0x01c5a8461d06c2c195035c148af0f871c7679841d86ae5bb98676bb2d8e68df
a","input":"0x","nonce":"0x2f31","to":"0xdae787ec66e65c60ad35203800b97b45fcb0f909","transactionIndex":"0x0","value":"0x2b81097c919a9e000","v":"0x1b","r":"0x1d263f32de00456cbc3e4099c623a6ca48e3188f01dd16461687512f8c3bb7d6","s":"0x62629f4
2a98f7c5fa60f8c3630cf67b5eb4772e0ff65a458d2116c15e22d6451"},{"blockHash":"0x9a834c53bbee9c2665a5a84789a1d1ad73750b2d77b50de44f457f411d02e52e","blockNumber":"0x100000","from":"0x68795c4aa09d6f4ed3e5deddf8c2ad3049a601da","gas":"0x15f90","
gasPrice":"0xba43b7400","hash":"0xb4216e88df6ebfe666bbe43370d1ddfe8d9c1975d2b8375a522700f7f59c3ed0","input":"0x","nonce":"0x2f32","to":"0x04a2a3ee9c9ea82e1f918ece209f8a7e6d7521e7","transactionIndex":"0x1","value":"0x377fb2f2cbd73c00","v
":"0x1c","r":"0xc25ceef654cd3bd734201fc08589970e0663431121bc47cc4a70335e072b09c0","s":"0x1c67b037a59a8f5badde3a7eb586d7888dcf6dad64aabb080c4c0db593a91254"}],"transactionsRoot":"0x91d9b419b34a790cc2a7d59c6b95e57ccc5ec9bc1034ce9510dfb551d
3049c93","uncles":[]}}
例子：一次代币的transfer交易（topics第一个值是sha3('Transfer(address,address,uint256)')，第二个值是源地址，第三个值是目标地址，data字段是以Wei为单位的转移的代币数目）
{"jsonrpc":"2.0","id":1,"result":{"blockHash":"0x0b7deffc8e3603870fe6caf6b0a3f7620cc4dd26c1950a25ce16dfbd94ff4ba1","blockNumber":"0x3f3711","contractAddress":null,"cumulativeGasUsed":"0x14da9b","from":"0x1c362ae991ff2b60c7f93eb1d2cc0237
51dcd8bc","gasUsed":"0xcbd5","logs":[{"address":"0x90528aeb3a2b736b780fd1b6c478bb7e1d643170","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x0000000000000000000000001c362ae991ff2b60c7f93eb1d2cc023751dcd
8bc","0x000000000000000000000000bf6631ba138efb863263bd90f80fb5117676a129"],"data":"0x0000000000000000000000000000000000000000000000001111d67bb1bb0000","blockNumber":"0x3f3711","transactionHash":"0x5c2bf2b66fd7996030fccbd711263d4ecd7c2c3
8909b747902c0df0504f71d90","transactionIndex":"0x3c","blockHash":"0x0b7deffc8e3603870fe6caf6b0a3f7620cc4dd26c1950a25ce16dfbd94ff4ba1","logIndex":"0x3","removed":false}],"logsBloom":"0x0000000000000000000000000000000002200000000004000000
00000000000000001000000000000000000000000000000000000000000000000000000000000000000000000020000000080000000000000000000000000000000000000000000000000000000800000000000000000000000000000000000000100000000000000000000000000000000000000000
00000000000000000000000000000000000000000400000000000000000000000000020000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000","root":"0x
66ccd3b82b30ae6f6622827e17f1d9845369ca732310fafe3be4475c032db322","to":"0x90528aeb3a2b736b780fd1b6c478bb7e1d643170","transactionHash":"0x5c2bf2b66fd7996030fccbd711263d4ecd7c2c38909b747902c0df0504f71d90","transactionIndex":"0x3c"}}