geth结构解析和源码分析

第一部分 看看geth客户端的整体结构
创建私链的时候已经指定所有的信息都放在private-geth目录下，现在是已经有过挖矿的目录。

当时我们把创世文件genesis.json放在该目录下了、

root@i-5tthrr8u:/home/ubuntu/private-geth# ll
total 16
drwxr-xr-x 3 root   root   4096 Jul  2 17:02 ./
drwxr-xr-x 6 ubuntu ubuntu 4096 Jul  4 14:07 ../
drwx------ 5 root   root   4096 Jul  2 17:41 data/
-rw-r--r-- 1 root   root    529 Jul  2 16:29 genesis.json

进入真正的存放数据的目录private-geth/data/00
geth中保存的是区块链的相关数据
keystore中保存的是该链条中的用户信息

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00# ll
total 20
drwx------ 4 root root 4096 Jul  2 17:23 ./
drwx------ 5 root root 4096 Jul  2 17:41 ../
drwxr-xr-x 5 root root 4096 Jul  2 17:02 geth/
-rw------- 1 root root 1391 Jul  2 17:58 history
drwx------ 2 root root 4096 Jul  2 17:10 keystore/

之前我们这个节点已经创建了两个账户，现在我们可以看到keystore里面有两个账户信息的文件

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/keystore# ll
total 16
drwx------ 2 root root 4096 Jul  2 17:10 ./
drwx------ 4 root root 4096 Jul  2 17:23 ../
-rw------- 1 root root  491 Jul  2 17:02 UTC--2017-07-02T09-02-56.470592674Z--28b769b3b9109afd1e9e50a9312c5a3bfae8a699
-rw------- 1 root root  491 Jul  2 17:10 UTC--2017-07-02T09-10-28.087401309Z--b4e2e2514eae3684157bf34a0cee2c07c431cf92

每个账户都由一对钥匙定义，一个私钥和一个公钥。 账户以地址为索引，地址由公钥衍生而来，取公钥的最后 20个字节。每对私钥 /地址都编码在一个钥匙文件里。钥匙文件是JSON文本文件，可以用任何文本编辑器打开和浏览。钥匙文件的关键部分，账户私钥，通常用你创建帐户时设置的密码进行加密。钥匙文件的文件名格式为UTC。账号列出时是按字母顺序排列，但是由于时间戳格式，实际上它是按创建顺序排列。如果把秘钥丢了钥匙文件可以在以太坊节点数据目录的keystore子目录下找到，接下来我们进入一个keystore目录文件看看他的信息：

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/keystore# vim UTC--2017-07-02T09-02-56.470592674Z--28b769b3b9109afd1e9e50a9312c5a3bfae8a699 

{"address":"28b769b3b9109afd1e9e50a9312c5a3bfae8a699",
"crypto":{
"cipher":"aes-128-ctr",
"ciphertext":"89ce1513b4b5a325735891b559c361ce696bb2c173a7a1b290549e79dad8f847",
"cipherparams":{"iv":"982c86418fae2dd39e04d1e51528cffa"},
"kdf":"scrypt",
"kdfparams":{"dklen":32,"n":262144,"p":1,"r":8,"salt":"4227384ea0e3d15af1bac190f7e01d392543d0a5ca1ec931c1d340f87845f771"},
"mac":"46cffc6e4f57fa27b69e53dc4ae43a03ce1b93f24c132aa4655f53ddf215f112"},
"id":"e516b9d4-2161-4648-b3db-fc2ef1c3739c",
"version":3
}

警告：记住密码并”备份钥匙文件”。为了从账号发送交易，包括发送以太币，你必须同时有钥匙文件和密码。确保钥匙文件有个备份并牢记密码，尽可能安全地存储它们。这里没有逃亡路径，如果钥匙文件丢失或忘记密码，就会丢失所有的以太币。没有密码不可能进入账号，也没有忘记密码选项。所以一定不要忘记密码。

接下来进入geth可以看到chaindata，lightchaindata，nodes目录

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth# ll
total 24
drwxr-xr-x 5 root root 4096 Jul  2 17:02 ./
drwx------ 4 root root 4096 Jul  2 17:23 ../
drwxr-xr-x 2 root root 4096 Jul  4 14:12 chaindata/
drwxr-xr-x 2 root root 4096 Jul  2 17:02 lightchaindata/
-rw-r--r-- 1 root root    0 Jul  2 17:02 LOCK
-rw------- 1 root root   64 Jul  2 17:02 nodekey
drwxr-xr-x 2 root root 4096 Jul  4 15:55 nodes/

进入nodes（我们这条私链有三个节点，所以这里有三个ldb文件）

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/nodes# ll
total 5316
drwxr-xr-x 2 root root    4096 Jul  4 15:55 ./
drwxr-xr-x 5 root root    4096 Jul  2 17:02 ../
-rw-r--r-- 1 root root  405250 Jul  4 15:57 000033.log
-rw-r--r-- 1 root root 2132979 Jul  4 15:55 000035.ldb
-rw-r--r-- 1 root root 2131238 Jul  4 15:55 000036.ldb
-rw-r--r-- 1 root root  739354 Jul  4 15:55 000037.ldb
-rw-r--r-- 1 root root      16 Jul  4 14:12 CURRENT
-rw-r--r-- 1 root root       0 Jul  2 17:02 LOCK
-rw-r--r-- 1 root root    8187 Jul  4 15:55 LOG
-rw-r--r-- 1 root root    4557 Jul  4 15:55 MANIFEST-000013

进入chaindata，区块链最后的本地存储都是以ldb文件的形势（但这里是不是应该每个区块一个ldb文件呢？）

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/chaindata# ll
total 52
drwxr-xr-x 2 root root  4096 Jul  5 09:51 ./
drwxr-xr-x 5 root root  4096 Jul  2 17:02 ../
-rw-r--r-- 1 root root  5288 Jul  2 17:56 000008.ldb
-rw-r--r-- 1 root root 11681 Jul  4 14:12 000009.ldb
-rw-r--r-- 1 root root  8921 Jul  4 14:13 000010.log
-rw-r--r-- 1 root root    16 Jul  4 14:12 CURRENT
-rw-r--r-- 1 root root     0 Jul  2 17:02 LOCK
-rw-r--r-- 1 root root  2807 Jul  4 14:12 LOG
-rw-r--r-- 1 root root   346 Jul  4 14:12 MANIFEST-000011

进入Lightchaindata

root@i-5tthrr8u:/home/ubuntu/private-geth/data/00/geth/lightchaindata# ll
total 24
drwxr-xr-x 2 root root 4096 Jul  2 17:02 ./
drwxr-xr-x 5 root root 4096 Jul  2 17:02 ../
-rw-r--r-- 1 root root 1237 Jul  2 17:02 000001.log
-rw-r--r-- 1 root root   16 Jul  2 17:02 CURRENT
-rw-r--r-- 1 root root    0 Jul  2 17:02 LOCK
-rw-r--r-- 1 root root  358 Jul  2 17:02 LOG
-rw-r--r-- 1 root root   54 Jul  2 17:02 MANIFEST-000000

第二部分 看看源码的结构

1 Core/types/block.go
首先看到的是一个区块的结构

// Block represents an entire block in the Ethereum blockchain.

type Block struct {
    header       *Header
    uncles       []*Header
    transactions Transactions

    // caches    hash和size字段是cache之用，避免多次 hash/sign导致性能损失
    hash atomic.Value
    size atomic.Value

    // Td is used by package core to store the total difficulty
    // of the chain up to and including the block.挖矿难度
    td *big.Int

    // These fields are used by package eth to track
    // inter-peer block relay.
    ReceivedAt   time.Time
    ReceivedFrom interface{}
}

这是一个区块体的结构，区块体是动态的存储数据的，主要包含了交易列表和uncle列表
// Body is a simple (mutable, non-safe) data container for storing and moving
// a block's data contents (transactions and uncles) together.
type Body struct {
    Transactions []*Transaction
    Uncles       []*Header
}

区块头的结构体，里面的参数我们都很熟悉就不解释了
// Header represents a block header in the Ethereum blockchain.
type Header struct {
    ParentHash  common.Hash    `json:"parentHash"       gencodec:"required"`
    UncleHash   common.Hash    `json:"sha3Uncles"       gencodec:"required"`
    Coinbase    common.Address `json:"miner"            gencodec:"required"`
    Root        common.Hash    `json:"stateRoot"        gencodec:"required"`
    TxHash      common.Hash    `json:"transactionsRoot" gencodec:"required"`
    ReceiptHash common.Hash    `json:"receiptsRoot"     gencodec:"required"`
    Bloom       Bloom          `json:"logsBloom"        gencodec:"required"`
    Difficulty  *big.Int       `json:"difficulty"       gencodec:"required"`
    Number      *big.Int       `json:"number"           gencodec:"required"`
    GasLimit    *big.Int       `json:"gasLimit"         gencodec:"required"`
    GasUsed     *big.Int       `json:"gasUsed"          gencodec:"required"`
    Time        *big.Int       `json:"timestamp"        gencodec:"required"`
    Extra       []byte         `json:"extraData"        gencodec:"required"`
    MixDigest   common.Hash    `json:"mixHash"          gencodec:"required"`
    Nonce       BlockNonce     `json:"nonce"            gencodec:"required"`
}

2 这是一个交易的结构体
Core/types/transaction.go

1ContractTransaction的区别在于：Recipient == nil ； 2. Transaction能以RLP算法进行Encode和Decode； 3. hash/size/from字段是cache之用，避免多次 hash/sign导致性能损失；
type Transaction struct {
    data txdata
    // caches
    hash atomic.Value
    size atomic.Value
    from atomic.Value
}

type txdata struct {
    AccountNonce uint64          `json:"nonce"    gencodec:"required"`
    Price        *big.Int        `json:"gasPrice" gencodec:"required"`
    GasLimit     *big.Int        `json:"gas"      gencodec:"required"`
    Recipient    *common.Address `json:"to"       rlp:"nil"` // nil means contract creation
    Amount       *big.Int        `json:"value"    gencodec:"required"`
    Payload      []byte          `json:"input"    gencodec:"required"`

    // Signature values 签名
    V *big.Int `json:"v" gencodec:"required"`
    R *big.Int `json:"r" gencodec:"required"`
    S *big.Int `json:"s" gencodec:"required"`

    // This is only used when marshaling to JSON.
    Hash *common.Hash `json:"hash" rlp:"-"`
}

3 Receiptroot我们刚刚在区块头有看到，那他具体包含的是什么呢？它是一个交易的结果，主要包括了poststate,交易所花费的gas,bloom和logs

// Receipt represents the results of a transaction.
type Receipt struct {
    // Consensus fields
    PostState         []byte   `json:"root"              gencodec:"required"`
    CumulativeGasUsed *big.Int `json:"cumulativeGasUsed" gencodec:"required"`
    Bloom             Bloom    `json:"logsBloom"         gencodec:"required"`
    Logs              []*Log   `json:"logs"              gencodec:"required"`

    // Implementation fields (don't reorder!)
    TxHash          common.Hash    `json:"transactionHash" gencodec:"required"`
    ContractAddress common.Address `json:"contractAddress"`
    GasUsed         *big.Int       `json:"gasUsed" gencodec:"required"`
}

4 一个个交易被打包到区块上面，那区块又是怎么变成去快链的呢？
Core/blockchain.go

// BlockChain represents the canonical chain given a database with a genesis block. The Blockchain manages chain imports, reverts, chain reorganisations.
// Importing blocks in to the block chain happens according to the set of rules defined by the two stage Validator. （需要两个阶段的验证）Processing of blocks is done using the Processor which processes the included transaction.（第一阶段交易的验证） The validation of the state is done in the second part of the Validator.（第二阶段state的验证） Failing results in aborting of the import.
// The BlockChain also helps in returning blocks from **any** chain included in the database as well as blocks that represents the canonical chain. It's important to note that GetBlock can return any block and does not need to be included in the canonical one where as GetBlockByNumber always represents the canonical chain.


type BlockChain struct {
    config *params.ChainConfig // chain & network configuration

    hc           *HeaderChain
    chainDb      **ethdb**.Database 本地数据库
    eventMux     *event.TypeMux
    genesisBlock *types.Block

    mu      sync.RWMutex // global mutex for locking chain operations
    chainmu sync.RWMutex // blockchain insertion lock
    procmu  sync.RWMutex // block processor lock

    checkpoint       int          // checkpoint counts towards the new checkpoint
    currentBlock     *types.Block // Current head of the block chain
    currentFastBlock *types.Block // Current head of the fast-sync chain (may be above the block chain!)

    stateCache   *state.StateDB // State database to reuse between imports (contains state cache)
    bodyCache    *lru.Cache     // Cache for the most recent block bodies
    bodyRLPCache *lru.Cache     // Cache for the most recent block bodies in RLP encoded format
    blockCache   *lru.Cache     // Cache for the most recent entire blocks
    futureBlocks *lru.Cache     // future blocks are blocks added for later processing

    quit    chan struct{} // blockchain quit channel
    running int32         // running must be called atomically
    // procInterrupt must be atomically called
    procInterrupt int32          // interrupt signaler for block processing
    wg            sync.WaitGroup // chain processing wait group for shutting down

    engine    consensus.Engine
    processor Processor // block processor interface
    validator Validator // block and state validator interface
    vmConfig  vm.Config

    badBlocks *lru.Cache // Bad block cache
}

注意：1. BlockChain无结构化查询需求，仅Hash查询， Key/Value数据库最方便； 2. 低层用LevelDB存储，性能好

5 stateDB用来存储世界状态
Core/state/statedb.go

// StateDBs within the ethereum protocol are used to store anything
// within the merkle trie. StateDBs take care of caching and storing
// nested states. It's the general query interface to retrieve:
// * Contracts
// * Accounts
type StateDB struct {
    db            ethdb.Database //本地数据库
    trie          *trie.SecureTrie
    pastTries     []*trie.SecureTrie
    codeSizeCache *lru.Cache

    // This map holds 'live' objects, which will get modified while processing a state transition.
    stateObjects           map[common.Address]*stateObject
    stateObjectsDirty      map[common.Address]struct{}
    stateObjectsDestructed map[common.Address]struct{}

    // The refund counter, also used by state transitioning.
    refund *big.Int

    thash, bhash common.Hash
    txIndex      int
    logs         map[common.Hash][]*types.Log
    logSize      uint

    preimages map[common.Hash][]byte

    // Journal of state modifications. This is the backbone of
    // Snapshot and RevertToSnapshot.
    journal        journal
    validRevisions []revision
    nextRevisionId int

    lock sync.Mutex
}

注意：1. StateDB完整记录Transaction的执行情况； 2. StateDB的重点是StateObjects； 3. StateDB中的 stateObjects，Account的Address为 key，记录其Balance、nonce、code、codeHash ，以及tire中的 {string:Hash}等信息；

那我们接下来看看stateObject结构体
Core/state/state_object.go

// stateObject represents an Ethereum account which is being modified.
//
// The usage pattern is as follows:
// First you need to obtain a state object.
// Account values can be accessed and modified through the object.
// Finally, call CommitTrie to write the modified storage trie into a database.
type stateObject struct {
    address common.Address // Ethereum address of this account
    data    Account
    db      *StateDB

    // DB error.
    // State objects are used by the consensus core and VM which are
    // unable to deal with database-level errors. Any error that occurs
    // during a database read is memoized here and will eventually be returned
    // by StateDB.Commit.
    dbErr error

    // Write caches.
    trie *trie.SecureTrie // storage trie, which becomes non-nil on first access
    code Code             // contract bytecode, which gets set when code is loaded

    cachedStorage Storage // Storage entry cache to avoid duplicate reads
    dirtyStorage  Storage // Storage entries that need to be flushed to disk

    // Cache flags.
    // When an object is marked suicided it will be delete from the trie
    // during the "update" phase of the state transition.
    dirtyCode bool // true if the code was updated
    suicided  bool
    touched   bool
    deleted   bool
    onDirty   func(addr common.Address) // Callback method to mark a state object newly dirty
}

再看看state的一个接口，可以查看账户的余额，nonce，代码和storage

// ChainStateReader wraps access to the state trie of the canonical blockchain. Note that implementations of the interface may be unable to return state values for old blocks.
// In many cases, using CallContract can be preferable to reading raw contract storage.

type ChainStateReader interface {
    BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)
    StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error)
    CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error)
    NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
}

所有的结构凑明朗了，那具体的验证过程是怎么样的呢
Core/state_processor.go
Core/state_transition.go
Core/block_validator.go

StateProcessor 1. 调用StateTransition，验证（执行）Transaction； 2. 计算Gas、Recipt、Uncle Reward

// StateProcessor is a basic Processor, which takes care of transitioning
// state from one point to another.
//
// StateProcessor implements Processor.
type StateProcessor struct {
    config *params.ChainConfig // Chain configuration options
    bc     *BlockChain         // Canonical block chain
    engine consensus.Engine    // Consensus engine used for block rewards
}

StateTransition
1. 验证（执行）Transaction；
3. 扣除transaction.data.payload计算数据所需要消耗的gas；
4. 在vm中执行code（生成contract or 执行contract）；vm执 行过程中，其gas会被自动消耗。如果gas不足，vm会自 选退出；
5. 将多余的gas退回到sender.balance中；
6. 将消耗的gas换成balance加到当前env.Coinbase()中；

/*
The State Transitioning Model

A state transition is a change made when a transaction is applied to the current world state
The state transitioning model does all all the necessary work to work out a valid new state root.

1) Nonce handling
2) Pre pay gas
3) Create a new state object if the recipient is \0*32
4) Value transfer
== If contract creation ==
  4a) Attempt to run transaction data
  4b) If valid, use result as code for the new state object
== end ==
5) Run Script section
6) Derive new state root
*/
type StateTransition struct {
    gp         *GasPool
    msg        Message
    gas        uint64
    gasPrice   *big.Int
    initialGas *big.Int
    value      *big.Int
    data       []byte
    state      vm.StateDB

    evm *vm.EVM
}

BlockValidator
1. 验证UsedGas
2. 验证Bloom
3. 验证receiptSha
4. 验证stateDB.IntermediateRoot

// BlockValidator is responsible for validating block headers, uncles and
// processed state.
//
// BlockValidator implements Validator.
type BlockValidator struct {
    config *params.ChainConfig // Chain configuration options
    bc     *BlockChain         // Canonical block chain
    engine consensus.Engine    // Consensus engine used for validating
}

可以注意到刚才的state和block都是写进db数据库的，那我们看一下leveldb数据库结构

type LDBDatabase struct {
    fn string      // filename for reporting
    db *leveldb.DB // LevelDB instance

    getTimer       gometrics.Timer // Timer for measuring the database get request counts and latencies
    putTimer       gometrics.Timer // Timer for measuring the database put request counts and latencies
    delTimer       gometrics.Timer // Timer for measuring the database delete request counts and latencies
    missMeter      gometrics.Meter // Meter for measuring the missed database get requests
    readMeter      gometrics.Meter // Meter for measuring the database get request data usage
    writeMeter     gometrics.Meter // Meter for measuring the database put request data usage
    compTimeMeter  gometrics.Meter // Meter for measuring the total time spent in database compaction
    compReadMeter  gometrics.Meter // Meter for measuring the data read during compaction
    compWriteMeter gometrics.Meter // Meter for measuring the data written during compaction

    quitLock sync.Mutex      // Mutex protecting the quit channel access
    quitChan chan chan error // Quit channel to stop the metrics collection before closing the database

    log log.Logger // Contextual logger tracking the database path
}

type Database interface {
    Put(key []byte, value []byte) error
    Get(key []byte) ([]byte, error)
    Delete(key []byte) error
    Close()
    NewBatch() Batch
}