Fabric多机部署-e2e

Fabric多机部署-e2e

测试环境为三台华为云ECS主机，加入在同一虚拟子网中；
实验将fabric/example/e2e_cli进行修改测试多机部署;
三个node上的fabric环境一致，需要修改e2e配置文件；
fabric v1.4已经移除e2e demo,本次版本为fabric v1.2。

• 部署环境

  node	org
  192.168.0.145	orderer、peer0.org1、peer1.org1、cli
  192.168.0.105	peer0.org2
  192.168.0.151	peer1.org2

• docker-compose文件修改

  • node1:修改base/docker-compose-base.yaml中orderer服务的配置,添加extra_hosts的org2配置

  27   orderer.example.com:
  28     container_name: orderer.example.com
  29     image: hyperledger/fabric-orderer
  30     extra_hosts: #
  31       - "peer0.org2.example.com:192.168.0.105"
  32       - "peer1.org2.example.com:192.168.0.151"
  33     environment:
  34       - ORDERER_GENERAL_LOGLEVEL=debug
  35       - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
  36       - ORDERER_GENERAL_GENESISMETHOD=file
      ...
  

  • node1:修改docker-compose-cli.yaml中cli的配置，并删除org2的服务配置

  123   cli:
  124     container_name: cli
  125     image: hyperledger/fabric-tools
  126     tty: true
  127     extra_hosts:
  128       - "peer0.org2.example.com:192.168.0.105"
  129       - "peer1.org2.example.com:192.168.0.151"
  130       #- "peer0.org1.example.com:192.168.0.145"
  131       #- "peer1.org1.example.com:192.168.0.145"
  132     environment:
  133       - GOPATH=/opt/gopath
  134       - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
  135       - CORE_LOGGING_LEVEL=DEBUG
  136       - CORE_PEER_ID=cli
  137       - CORE_PEER_ADDRESS=peer0.org1.example.com:7051
  138       - CORE_PEER_LOCALMSPID=Org1MSP
      ...
  

  • node2:修改docker-compose-cli.yaml中peer0.org2的配置，删除其他服务及peer1.org2的配置

   8 services:
   9   peer0.org2.example.com:
  10     container_name: peer0.org2.example.com
  11     extends:
  12       file:  base/docker-compose-base.yaml
  13       service: peer0.org2.example.com
  14     extra_hosts:
  15       - "orderer.example.com:192.168.0.145"
  16       - "peer1.org2.example.com:192.168.0.151"
       
       
  

  • node2:修改base/docker-compose-base.yaml中peer0.org2的端口

   98   peer0.org2.example.com:
   99     container_name: peer0.org2.example.com
  100     extends:
  101       file: peer-base.yaml
  102       service: peer-base
  103     environment:
  104       - CORE_PEER_ID=peer0.org2.example.com
  105       - CORE_PEER_ADDRESS=peer0.org2.example.com:7051
  106       - CORE_PEER_CHAINCODEADDRESS=peer0.org2.example.com:7052
  107       - CORE_PEER_CHAINCODELISTENADDRESS=0.0.0.0:7052
  108       - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.org2.example.com:7051
  109       - CORE_PEER_LOCALMSPID=Org2MSP
  110     volumes:
  111         - /var/run/:/host/var/run/
  112         - ../crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp:/etc/hyperledger/fabric/msp
  113         - ../crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls:/etc/hyperledger/fabric/tls
  114     ports:
  115       - 7051:7051
  116       - 7052:7052
  117       - 7053:7053
  

  • node3:修改docker-compose-cli.yaml中peer1.org2的配置，删除其他服务及peer0.org2的配置

   8 services:
   9   peer1.org2.example.com:
  10     container_name: peer1.org2.example.com
  11     extends:
  12       file:  base/docker-compose-base.yaml
  13       service: peer1.org2.example.com
  14     extra_hosts:
  15       - "orderer.example.com:192.168.0.145"
  16       - "peer0.org2.example.com:192.168.0.105"
  

  • node3：修改base/docker-compose-base.yaml中peer1.org2的端口

  119   peer1.org2.example.com:
  120     container_name: peer1.org2.example.com
  121     extends:
  122       file: peer-base.yaml
  123       service: peer-base
  124     environment:
  125       - CORE_PEER_ID=peer1.org2.example.com
  126       - CORE_PEER_ADDRESS=peer1.org2.example.com:7051
  127       - CORE_PEER_CHAINCODEADDRESS=peer1.org2.example.com:7052
  128       - CORE_PEER_CHAINCODELISTENADDRESS=0.0.0.0:7052
  129       - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.org2.example.com:7051
  130       - CORE_PEER_GOSSIP_BOOTSTRAP=peer0.org2.example.com:7051
  131       - CORE_PEER_LOCALMSPID=Org2MSP
  132     volumes:
  133         - /var/run/:/host/var/run/
  134         - ../crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/msp:/etc/hyperledger/fabric/msp
  135         - ../crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls:/etc/hyperledger/fabric/tls
  136     ports:
  137       - 7051:7051
  138       - 7052:7052
  139       - 7053:7053
  

• 启动前，生成初始块及相关证书

  简化操作，注释掉三个node上network_setup.sh脚本的networkDown函数中删除证书和初始块的命令
  

  77 function networkDown () {
  78     docker-compose -f $COMPOSE_FILE -f $COMPOSE_FILE_COUCH down
  79 
  80     #Cleanup the chaincode containers
  81     clearContainers
  82 
  83     #Cleanup images
  84     removeUnwantedImages
  85 
  86     # remove orderer block and other channel configuration transactions and certs
  87     #rm -rf channel-artifacts/*.block channel-artifacts/*.tx crypto-config
  88 }   
  

  ./network_setup.sh启动，生成channel-artifacts、crypto-config两个文件，通过scp命令服务到node2、node3的e2e目录下
  

• 测试

  启动demo,在三个node上启动脚本
  

  #node1
  root@liu:~/work/go/src/github.com/hyperledger/fabric/examples/e2e_cli# ./network_setup.sh 
  #node2
  root@lhys:~/work/go/src/github.com/hyperledger/fabric/examples/e2e_cli# ./network_setup.sh 
  #node3
  [root@cce-demo1522483688765-00274 e2e_cli]# ./network_setup.sh 
  

  观察node1的输出，正常结果如下所示
  

...
2018-12-27 02:50:36.649 UTC [channelCmd] executeJoin -> INFO 041 Successfully submitted proposal to join channel
===================== peer0.org2 joined channel 'mychannel' =====================  
...
2018-12-27 02:50:44.068 UTC [channelCmd] update -> INFO 04d Successfully submitted channel update
===================== Anchor peers updated for org 'Org2MSP' on channel 'mychannel' ===================== 
...
2018-12-27 02:50:49.607 UTC [chaincodeCmd] install -> INFO 083 Installed remotely response:<status:200 payload:"OK" > 
===================== Chaincode is installed on peer0.org2 ===================== 
...
===================== All GOOD, End-2-End execution completed ===================== 


 _____   _   _   ____            _____   ____    _____ 
| ____| | \ | | |  _ \          | ____| |___ \  | ____|
|  _|   |  \| | | | | |  _____  |  _|     __) | |  _|  
| |___  | |\  | | |_| | |_____| | |___   / __/  | |___ 
|_____| |_| \_| |____/          |_____| |_____| |_____|

中间如果有其他异常，比如org2拒绝连接，背书失败等，可通过docker log 查看日志

• 测试过程中遇到的问题

  • ip记错，导致org2 connect: connection refused
  • 端口没改，因为e2e中peer均开方7051、7052、7053端口，为防止冲突，fabric进行了暴露了对应的外部端口如80** 90** 100**；
    但peer内部访问仍是705*,单机下fabric进行了转发处理，多机下需要改回705*端口

• 总结

  • 添加为orderer、org2、cli添加extra_hosts
  • 修改org2的peer容器暴露的端口
  • 生成创世块、证书，并复制到其他节点