Hyperledger caliper架构介绍和源码分析

简要分析caliper 的代码实现，以 benchmark/simple为例。
根据 caliper架构图：

Application layer

应用程序层包含针对典型区块链方案实施的测试。 每个测试都有一个配置文件，用于定义后端区块链网络和测试参数。 这些测试可以直接用于测试区块链系统的性能。
开发人员可以直接使用NBI来实现自己的测试而不需要框架。

Adaptation layer

适配层主要作用是将现有的区块链系统集成到caliper框架中。每个适配器使用相应的区块链原生（native）SDK或RESTful API实现 ‘Caliper Blockchain NBI’。
现在支持Hyperledger Fabric1.0 和 Sawtooth，以太坊和其他的区块链系统仍在计划中。
src文件夹下为该项目的主要代码。
src/comm包内是Interface & Core Layer的代码；
src/fabric、src/sawtooth、src/iroha包内是Adaptation layer的具体代码。

fabric.js中的Fabric类就继承自BlockchainInterface，实现了初始化、安装/执行智能合约等功能。
这些功能内部自然就是调用了fabric-client，所以测试前需要先进行安装：

npm install grpc@1.10.1 fabric-ca-client fabric-client

Benchmark layer

测试用例位于benchmark/目录下， 由测试人员编写，并配置到-c指定的配置文件的test.rounds[.callback]中。
每个测试用例都包含定义了具体功能的.js文件（如main.js，query.js）和两种配置文件（config.json和fabric.json）。如下为marbles示例：

Configuration file

使用两种配置文件：
config.json：benchmark配置文件，定义benchmark测试的参数，如测试轮次，工作负载等；
fabric.json：区块链配置文件，指定了与SUT（system under test）交互的必要信息，即区块链网络的配置，如peer数，client数等。

这两个配置文件很重要，caliper就是根据这两个配置文件的设置来进行测试的。

• benchmark configuration file
  example：

{
  "blockchain": {
    "type": "fabric",
    "config": "./fabric.json"
  },
  "command" : {
    "start": "docker-compose -f ../../network/fabric/simplenetwork/docker-compose.yaml up -d",
    "end" : "docker-compose -f ../../network/fabric/simplenetwork/docker-compose.yaml down;docker rm $(docker ps -aq)"
  },
  "test": {
    "name": "simple",
    "description" : "This is an example benchmark for caliper",
    "clients": {
      "type": "local",
      "number": 5
    },
    "rounds": [{
        "label" : "open",
        "txNumber" : [5000, 5000, 5000],
        "rateControl" : [{"type": "fixed-rate", "opts": {"tps" : 100}}, {"type": "fixed-rate", "opts": {"tps" : 200}}, {"type": "fixed-rate", "opts": {"tps" : 300}}],
        "arguments": {  "money": 10000 },
        "callback" : "benchmark/simple/open.js"
      },
      {
        "label" : "query",
        "txNumber" : [5000, 5000],
        "rateControl" : [{"type": "fixed-rate", "opts": {"tps" : 300}}, {"type": "fixed-rate", "opts": {"tps" : 400}}],
        "callback" : "benchmark/simple/query.js"
      }]
  },
  "monitor": {
    "type": ["docker", "process"],
    "docker":{
      "name": ["peer0.org1.example.com", "http://192.168.1.100:2375/orderer.example.com"]
    },
    "process": [
      {
        "command" : "node",
        "arguments" : "local-client.js",
        "multiOutput" : "avg"
      }
    ],
    "interval": 1
  }
}

• Fabric Configuration file
  example

{
  "fabric": {
    "cryptodir": "network/fabric/simplenetwork/crypto-config",
    "network": {
      "orderer": {
        "url": "grpcs://localhost:7050",
        "mspid": "OrdererMSP",
        "domain": "example.com",
        "user": {
          "key": "network/fabric/simplenetwork/crypto-config/ordererOrganizations/example.com/users/[email protected]/msp/keystore/be595291403ff68280a724d7e868521815ad9e2fc8c5486f6d7ce6b62d6357cd_sk",
          "cert": "network/fabric/simplenetwork/crypto-config/ordererOrganizations/example.com/users/[email protected]/msp/signcerts/[email protected]"
        },
        "server-hostname": "orderer.example.com",
        "tls_cacerts": "network/fabric/simplenetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt"
      },
      "org1": {
        "name": "peerOrg1",
        "mspid": "Org1MSP",
        "domain": "org1.example.com",
        "user": {
          "key": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org1.example.com/users/[email protected]/msp/keystore/0d2b2fc385b10fa59003217e1bb5af2d24a3d762266e287867a1bc290eb44657_sk",
          "cert": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org1.example.com/users/[email protected]/msp/signcerts/[email protected]"
        },
        "ca": {
          "url": "https://localhost:7054",
          "name": "ca-org1"
        },
        "peer1": {
          "requests": "grpcs://localhost:7051",
          "events": "grpcs://localhost:7053",
          "server-hostname": "peer0.org1.example.com",
          "tls_cacerts": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt"
        },
        "peer2": {
          "requests": "grpcs://localhost:7057",
          "events": "grpcs://localhost:7059",
          "server-hostname": "peer1.org1.example.com",
          "tls_cacerts": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls/ca.crt"
        }
      },
      "org2": {
        "name": "peerOrg2",
        "mspid": "Org2MSP",
        "domain": "org2.example.com",
        "ca": {
          "url": "https://localhost:8054",
          "name": "ca-org2"
        },
        "peer1": {
          "requests": "grpcs://localhost:8051",
          "events": "grpcs://localhost:8053",
          "server-hostname": "peer0.org2.example.com",
          "tls_cacerts": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt"
        },
        "peer2": {
          "requests": "grpcs://localhost:8057",
          "events": "grpcs://localhost:8059",
          "server-hostname": "peer1.org2.example.com",
          "tls_cacerts": "network/fabric/simplenetwork/crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/ca.crt"
        }
      }
    },
    "channel": [
      {
        "name": "mychannel",
        "config": "network/fabric/simplenetwork/mychannel.tx",
        "organizations": ["org1", "org2"],
        "deployed": false
      }
    ],
    "chaincodes": [{"id": "simple", "path": "contract/fabric/simple/go", "language":"golang", "version": "v0", "channel": "mychannel"}],
    "endorsement-policy": {
      "identities": [
        {
          "role": {
            "name": "member",
            "mspId": "Org1MSP"
          }
        },
        {
          "role": {
            "name": "member",
            "mspId": "Org2MSP"
          }
        },
        {
          "role": {
            "name": "admin",
            "mspId": "Org1MSP"
          }
        }
      ],
      "policy": { "2-of": [{"signed-by": 0}, {"signed-by": 1}]}
    },
    "context": {
      "open": "mychannel",
      "query": "mychannel"
    }
  },
  "info" : {
    "Version": "1.0.5",
    "Size": "4 Peers",
    "Orderer": "Solo",
    "Distribution": "Single Host"
  }
}

fabric测试流程：

main.js

位置：benchmark/simple/main.js

运行测试时执行：

 node benchmark/simple/main.js -c yourconfig.json -n yournetwork.json

其中，main.js主要加载两个配置文件config.json和fabric.json：

/**
 * Set benchmark config file
 * @param {*} file config file of the benchmark,  default is config.json
 */
function setConfig(file) {
    configFile = file;
}

/**
 * Set benchmark network file
 * @param {*} file config file of the blockchain system, eg: fabric.json
 */
function setNetwork(file) {
    networkFile = file;
}

然后调用了src/comm/bench-flow.js的run方法，将两个配置文件传进去:

    const framework = require('../../src/comm/bench-flow.js');
    framework.run(absConfigFile, absNetworkFile);

bench-flow.js（in interface&core layer）：

位置：src/comm/bench-flow.js
编写了生成html report的函数，并根据配置文件中定义的内容进行测试。

const Blockchain = require('./blockchain.js');
const Monitor = require('./monitor.js');
const Report  = require('./report.js');
const Client  = require('./client/client.js');
const Util = require('./util.js');

/**
 * Start a default test flow to run the tests
 * @param {String} configFile path of the test configuration file
 * @param {String} networkFile path of the blockchain configuration file
 */
module.exports.run = function(configFile, networkFile) {
    test('#######Caliper Test######', (t) => {
        global.tapeObj = t;
        absConfigFile  = Util.resolvePath(configFile);
        absNetworkFile = Util.resolvePath(networkFile);
        blockchain = new Blockchain(absNetworkFile);
        monitor = new Monitor(absConfigFile);
        client  = new Client(absConfigFile);
        createReport();
        demo.init();
        ......
        }
      }

run方法定义了完整的测试流程，包括初始化区块链：

startPromise.then(() => {
        return blockchain.init();
    }).

安装智能合约：

then( () => {
        return blockchain.installSmartContract();
    }).

初始化客户端，在区块链上准备指定数量的客户端：

then( () => {
        return client.init().then((number)=>{
            return blockchain.prepareClients(number);
        });
    }).

对资源进行监视：

then( (clientArgs) => {
        monitor.start().then(()=>{
           .....
    }).

打印检测结果，关闭监视器：

then( () => {
        log('----------finished test----------\n');
        printResultsByRound();
        monitor.printMaxStats();
        monitor.stop();
           ...
    }).

关闭客户端：

then( () => {
        client.stop();
            ...
    }).

Benchmark Engine

Benchmark Engine位于 Benchmark层，通过配置文件将用户编写的测试用例集成到caliper框架中。

1.配置文件（Configuration File）

2.Master

Master实现了默认的测试流程，包括3个阶段：

• 准备阶段（Preparing stage）：master使用区块链配置文件创建并初始化内部区块链对象（internal blockchain object），部署配置中指定的智能合约，并启动监控对象以监控后端区块链系统的资源消耗。
• 测试阶段（Testing stage）：master根据benchmark 配置文件启动循环以执行测试。 将根据定义的工作负载生成任务（tasks）并将其分配给客户端。将存储客户端返回的性能统计信息以供以后分析。
• 报告阶段（Reporting stage）：分析每个测试轮次的所有clients的统计数据，并自动生成HTML格式报告。

Interface & Core Layer

接口&核心层实现了该工具的核心函数，并为up-applications提供了north bound interfaces（NBI），共四种NBIs:

• 区块链操作接口（Blockchain operating interfaces）：包括在后端区块链上部署智能合约、调用合约、从账本查询状态等操作。
• 资源监控器（Resource Monitor）：包括启动/停止监控器以及获取后端区块链系统的资源消耗状态等操作（包括CPU，内存，网络IO等）。目前提供了两种监控器，一种用于监控本地/远程的（ local/remote） docker 容器，另一种用于监控本地进程（local processes）。
• 性能分析器（Performance Analyzer）：包括读取预定义的性能统计数据（TPS，延迟，成功率等）和打印benchmark结果的操作。在调用区块链的NBIs会记录关键指标（key metrics ），例如交易的创建时间和提交时间，交易结果等。这些指标将用于稍后生成统计数据。
• 报告生成器（Report Generator）：包括用于生成HTML 格式的测试报告的操作。

src/comm目录下定义了非常多的内容，包括接口、监视器、报告等，整体目录如下：

blockchain-interface.js

位置：src/comm/blockchain-interface.js
其中的BlockchainInterface类定义了区块链的基本操作：

//Constructor
constructor(configPath) {
    this.configPath = configPath;
}
//初始化测试环境
init() {
    throw new Error('init is not implemented for this blockchain system');
}
//安装智能合约
installSmartContract() {
    throw new Error('installSmartContract is not implemented for this blockchain system');
}
//为测试客户端执行必要的准备
prepareClients (number) {
    let result = [];
    for(let i = 0 ; i< number ; i++) {
        result[i] = {}; // as default, return an empty object for each client
    }
    return Promise.resolve(result);
}
//获取后续操作的上下文
//返回的上下文对象的'engine'属性必须保留给benchm引擎以扩展上下文
getContext(name, args) {
    throw new Error('getContext is not implemented for this blockchain system');
}
//发布上下文以及相关资源
releaseContext(context) {
    throw new Error('releaseContext is not implemented for this blockchain system');
}
//调用智能合约
invokeSmartContract(context, contractID, contractVer, args, timeout) {
    throw new Error('invokeSmartContract is not implemented for this blockchain system');
}
//从账本中查询状态
 queryState(context, contractID, contractVer, key) {
    throw new Error('queryState is not implemented for this blockchain system');
}
//获取适配器特定的交易统计(Get adapter specific transaction statistics)
getDefaultTxStats(stats, results) {
    throw new Error('getDefaultTxStats is not implemented for this blockchain system');
}

monitor.js

位置：src/comm/monitor.js
其中的类class Monitor 定义了监视指定目标的资源消耗的操作。包括对监视器进行的start()， stop() ， restart()，getDefaultStats()（获取默认统计信息表） ，printMaxStats() （打印所有已监视项目的最大值）等函数。

monitor-interface.js

位置：src/comm/monitor-interface.js
类似于blockchain-interface.js，该文件定义了监视器的基本操作：

 //Constructor
    constructor(filter, interval) {
        this.filter       = filter;
        this.interval     = interval*1000; // ms
    }

    //start monitoring
    start() {
        throw new Error('start is not implemented for this monitor');
    }

    //restart monitoring
    restart() {
        throw new Error('restart is not implemented for this monitor');
    }

    //stop monitoring
    stop() {
        throw new Error('stop is not implemented for this monitor');
    }

    //Get watching list
    getPeers() {
        throw new Error('getPeers is not implemented for this monitor');
    }

    //Get history of memory usage, in byte
    getMemHistory(key) {
        throw new Error('getMemHistory is not implemented for this monitor');
    }

    //Get history of cpu usage, %
    getCpuHistory(key) {
        throw new Error('getCpuHistory is not implemented for this monitor');
    }

    //Get history of network IO usage, byte
    getNetworkHistory(key) {
        throw new Error('getCpuHistory is not implemented for this monitor');
    }

可以看出主要对内存、cpu占有率和网络IO进行了监控。
另外，src/comm下还有monitor-docker.js和monitor-process.js两个文件，是分别对docker和process资源监控的实现。
monitor-docker.js包括主要方法：

//Find local containers according to searching filters
function findContainers() {....}

和主要类：

 //Resource monitor for local/remote docker containers
class MonitorDocker extends MonitorInterface {
    //main functions:
    
    //Start the monitor
    start() {...}
    //Restart the monitor
    restart() {...}
    //Stop the monitor
    stop() {...}
    //Get information of watched containers
    getPeers() {...}
    //Get history of memory usage
    getMemHistory(key) {...}
    //Get history of CPU usage
    getCpuHistory(key) {...}
    //Get history of network IO usage as {in, out}
    getNetworkHistory(key) {...}
 }

monitor-process.js类似。

blockchain.js

位置：src/comm/blockchain.js
其中的class Blockchain定义了与被测区块链系统进行交互的操作：

    //Constructor
    constructor(configPath) {...}
    //return the blockchain's type
    gettype() {...}
    //Initialise test environment, 比如为test创建一个fabric channel
    init() {...}
    //为test clients执行必要的准备，例如 注册clients并获得密钥对
    prepareClients (number) {...}
    // Install smart contract(s), 详细信息已经在区块链配置文件中定义
    installSmartContract() {...}
    //Get a context for subsequent operations, 例如调用智能合约或查询state
    getContext(name, args) {...}
    //Release a context as well as related resources(发布上下文及相关资源)
    releaseContext(context) {...}
    // Invoke smart contract/submit transactions and return corresponding transactions' status
    invokeSmartContract(context, contractID, contractVer, args, timeout) {...}
    // Query state from the ledger
    queryState(context, contractID, contractVer, key) {...}
    //Calculate the default transaction statistics
    getDefaultTxStats(results, detail) {...}
    // merge an array of default 'txStatistics', the result is in first object of the array
    // Note even failed the first object of the array may still be changed
    static mergeDefaultTxStats(results) {...}

report.js

位置：src/comm/report.js
其中的类class Report用于统计数据和生成test report。

transaction.js

位置：src/comm/transaction.js
其中的类class TxStatus是Caliper的内部事务状态类，定义了交易的getter()和setter()方法。

Reference：
https://hyperledger.github.io/caliper/docs/2_Architecture.html
https://blog.csdn.net/get_set/article/details/81055220