hyperledger-fabric 的官方文档里没有针对多节点的配置,只给出了单机网络的部署方法。不知道是不是觉得程序员弄懂了单机天然就会部署多机器?一直想着写个教程,拖到这会儿,发现实施起来还挺费劲。
本文以官方文档https://hyperledger-fabric.readthedocs.io/en/release-1.4/build_network.html 为参考,尝试将其中介绍的fabric-samples/first-network样例部署在两台机器上。为简单起见,本例完全与官网文档中的first-network相同,只是把原来都放在一台机器上的5个实例分散在两台机器:
node1: orderer.example.com, peer0.org1.example.com, peer1.org1.example.com;
node2: peer0.org2.example.com, peer1.org2.example.com
建议一步一步走,坑不少。
su - root
密码:Root1234
# Uninstall old versions
sudo yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
# SET UP THE REPOSITORY
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# INSTALL DOCKER ENGINE - COMMUNITY
# Install the latest version of Docker Engine - Community and containerd
sudo yum install docker-ce docker-ce-cli containerd.io
# Start Docker.
sudo systemctl start docker
# Verify that Docker Engine - Community is installed correctly by running the hello-world image.
sudo docker run hello-world
约需要十几分钟(过程有确认提示,确认即可)。
# Create the docker group.
$ sudo groupadd docker
# Add your user to the docker group.
sudo usermod -aG docker duoduo
# Log out and log back in so that your group membership is re-evaluated.
exit #exit root
exit #exit sduoduossh 重新登陆
# verify
[saduoduohengda-13-vm3 ~]$ docker run hello-world
正常运行,成功!
在macOS下通过下载dmg的方式安装会附带docker-compose,但是linux按以上方式安装没有附带,需要单独安装。
参考官网 https://docs.docker.com/compose/install/
# 下载当前最新稳定版,这会需要几十分钟或数小时,如果太慢或遇到网络断开可尝试后一种方法
su - root
curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
#添加执行权限
sudo chmod +x /usr/local/bin/docker-compose
备选:如果是相同系统的多个节点,其中一台机子已经安装成功的前提下可以从该拷贝,直接将源机器的/usr/local/bin/docker-compose 拷贝到目标机的相同路径,然后参照上述代码添加执行权限。
参考 https://golang.org/doc/install
cd /home/duoduo/app
wget https://dl.google.com/go/go1.13.5.linux-amd64.tar.gz
su - root
tar -C /usr/local -xzf go1.13.5.linux-amd64.tar.gz
vi /etc/profile
追加
# set Gopath
export PATH=$PATH:/usr/local/go/bin
# 使其生效
source /etc/profile
# 验证
go version
打印go version go1.13.5 linux/amd64
vi test.go 写一段简单代码:
package main
import "fmt"
func main() {
fmt.Printf("hello, world\n")
}
# 保存,编译,执行
build test.go
./test
打印hello, world 成功!
cd app
curl -sSL https://bit.ly/2ysbOFE | bash -s
#verify
cd fabric-samples/first-network
./byfn up
curl指令执行很可能失败,需要将https://bit.ly/2ysbOFE放到浏览器中打开,会看到一个sh文件内容,拷贝下来存储为bootstrap.sh(放到app路径下);然后执行
sh bootstrap.sh
其中downloading binary很大概率会失败,从日志中将链接拷贝出来直接用放浏览器地址栏然后回车下载会比较快;下载完以后传到服务器的app路径下
scp ~/Downloads/hyperledger-xxxxxx/tar.gz node1:/home/duoduo/app
然后修改bootstrap.sh
...
# This will download the .tar.gz
download() {
local BINARY_FILE=$1
local URL=$2
echo "===> Downloading: " "${URL}"
#curl -s -L "${URL}" | tar xz || rc=$? #注释掉这行,添加以下两行
# 添加两行
cp /home/duoduo/app/${BINARY_FILE} ./
tar zxvf ${BINARY_FILE} || rc=$?
if [ -n "$rc" ]; then
echo "==> There was an error downloading the binary file."
return 22
else
echo "==> Done."
fi
}
重新执行bootstrap.sh:
sh bootstrap.sh
su - root
vi /etc/docker/daemon.json
添加内容(来自mac的配置,不行在网上多找几个镜像源)
{
"registry-mirrors" : [
"https://docker.mirrors.ustc.edu.cn",
"https://hub-mirror.c.163.com"
],
"debug" : true,
"experimental" : false
}
#restart
sudo service docker restart
现在本机将image导出为tar
docker images
docker save hyperledger/fabric-ca:1.4.4 -o fabric-ca.1.4.4.tar
...
# 发送至远端服务器
scp fabric-*.tar node1:/home/duoduo/app/
在远端导入:
docker load -i /home/duoduo/app/fabric-ca.1.4.4.tar
...
经过上述步骤,此时hyperledger-fabric文档中示例的first-network应该能起来了,验证一下:
cd /home/duoduo/app/fabric-samples/first-network
./byfn up
一连串执行日志,最后打印successful,这需要几分钟,日志类似如下:
Querying chaincode on peer1.org2...
===================== Querying on peer1.org2 on channel 'mychannel'... =====================
+ peer chaincode query -C mychannel -n mycc -c '{"Args":["query","a"]}'
Attempting to Query peer1.org2 ...3 secs
+ res=0
+ set +x
90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================
========= All GOOD, BYFN execution completed ===========
_____ _ _ ____
| ____| | \ | | | _ \
| _| | \| | | | | |
| |___ | |\ | | |_| |
|_____| |_| \_| |____/
见前述步骤。
这里参考https://hyperledger-fabric.readthedocs.io/en/release-1.4/build_network.html的后半部分尝试探索,即这段话之后的内容:
If you’d like to learn more about the underlying tooling and bootstrap mechanics, continue reading. In these next sections we’ll walk through the various steps and requirements to build a fully-functional Hyperledger Fabric network.
参考 https://hyperledger-fabric.readthedocs.io/en/release-1.4/build_network.html#manually-generate-the-artifacts。
cd fabric-samples/first-network
./byfn.sh down
以下步骤选择一个节点做即可。
$ ../bin/cryptogen generate --config=./crypto-config.yaml
org1.example.com
org2.example.com
# 先配置环境变量,后续指令会用到
export FABRIC_CFG_PATH=$PWD
# 按配置生成创世块
../bin/configtxgen -profile SampleMultiNodeEtcdRaft -channelID byfn-sys-channel -outputBlock ./channel-artifacts/genesis.block
export CHANNEL_NAME=mychannel && ../bin/configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID $CHANNEL_NAME
../bin/configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org1MSP
../bin/configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org2MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org2MSP
cd fabric-samples/first-network
scp -r channel-artifacts 172.16.1.4:/home/duoduo/app/fabric-samples/first-network
scp -r crypto-config 172.16.1.4:/home/duoduo/app/fabric-samples/first-network
# docker-compose-cli.yaml 直接在官方样例基础上改,注意中文注释的地方
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
orderer.example.com:
peer0.org1.example.com:
peer1.org1.example.com:
# 注释掉所有org2的配置,这两个实例运行在node2中
# peer0.org2.example.com:
# peer1.org2.example.com:
networks:
byfn:
services:
orderer.example.com:
extends:
file: base/docker-compose-base.yaml
service: orderer.example.com
container_name: orderer.example.com
networks:
- byfn
peer0.org1.example.com:
container_name: peer0.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org1.example.com
networks:
- byfn
# 添加外部主机映射,因为peer0.org2,peer1.org2两个实例都在node2,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "peer0.org2.example.com:172.16.1.4"
- "peer1.org2.example.com:172.16.1.4"
peer1.org1.example.com:
container_name: peer1.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer1.org1.example.com
networks:
- byfn
# 添加外部主机映射,因为peer0.org2,peer1.org2两个实例都在node2,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "peer0.org2.example.com:172.16.1.4"
- "peer1.org2.example.com:172.16.1.4"
# 注释掉所有org2的配置,这两个实例运行在node2中
# peer0.org2.example.com:
# container_name: peer0.org2.example.com
# extends:
# file: base/docker-compose-base.yaml
# service: peer0.org2.example.com
# networks:
# - byfn
# peer1.org2.example.com:
# container_name: peer1.org2.example.com
# extends:
# file: base/docker-compose-base.yaml
# service: peer1.org2.example.com
# networks:
# - byfn
cli:
container_name: cli
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- SYS_CHANNEL=$SYS_CHANNEL
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- FABRIC_LOGGING_SPEC=DEBUG
- FABRIC_LOGGING_SPEC=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
depends_on:
- orderer.example.com
- peer0.org1.example.com
- peer1.org1.example.com
# 注释掉所有org2的配置,这两个实例运行在node2中 这里cli其实只要能连上peer0.org1就可以了
# - peer0.org2.example.com
# - peer1.org2.example.com
networks:
- byfn
# 添加外部主机映射,因为peer0.org2,peer1.org2两个实例都在node2,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "peer0.org2.example.com:172.16.1.4"
- "peer1.org2.example.com:172.16.1.4"
node2 - 172.16.1.4
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
# 注释掉orderer和org1的实例配置,它们运行在node1
# orderer.example.com:
# peer0.org1.example.com:
# peer1.org1.example.com:
peer0.org2.example.com:
peer1.org2.example.com:
networks:
byfn:
services:
# 注释掉orderer和org1的实例配置,它们运行在node1
# orderer.example.com:
# extends:
# file: base/docker-compose-base.yaml
# service: orderer.example.com
# container_name: orderer.example.com
# networks:
# - byfn
# peer0.org1.example.com:
# container_name: peer0.org1.example.com
# extends:
# file: base/docker-compose-base.yaml
# service: peer0.org1.example.com
# networks:
# - byfn
# peer1.org1.example.com:
# container_name: peer1.org1.example.com
# extends:
# file: base/docker-compose-base.yaml
# service: peer1.org1.example.com
# networks:
# - byfn
peer0.org2.example.com:
container_name: peer0.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org2.example.com
networks:
- byfn
# 添加外部主机映射,因为orderer,peer0.org1,peer1.org1三个实例都在node1,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "orderer.example.com:172.16.1.3"
- "orderer2.example.com:172.16.1.3"
- "orderer3.example.com:172.16.1.3"
- "orderer4.example.com:172.16.1.3"
- "orderer5.example.com:172.16.1.3"
- "peer0.org1.example.com:172.16.1.3"
- "peer1.org1.example.com:172.16.1.3"
peer1.org2.example.com:
container_name: peer1.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer1.org2.example.com
networks:
- byfn
# 添加外部主机映射,因为orderer,peer0.org1,peer1.org1三个实例都在node1,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "orderer.example.com:172.16.1.3"
- "orderer2.example.com:172.16.1.3"
- "orderer3.example.com:172.16.1.3"
- "orderer4.example.com:172.16.1.3"
- "orderer5.example.com:172.16.1.3"
- "peer0.org1.example.com:172.16.1.3"
- "peer1.org1.example.com:172.16.1.3"
cli:
container_name: cli
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- SYS_CHANNEL=$SYS_CHANNEL
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- FABRIC_LOGGING_SPEC=DEBUG
- FABRIC_LOGGING_SPEC=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
depends_on:
# 注释掉前三个实例,cli的运行不需要依赖它们
# - orderer.example.com
# - peer0.org1.example.com
# - peer1.org1.example.com
- peer0.org2.example.com
- peer1.org2.example.com
networks:
- byfn
# 添加外部主机映射,因为orderer,peer0.org1,peer1.org1三个实例都在node1,需要告诉本实例怎么映射它们的域名
extra_hosts:
- "orderer.example.com:172.16.1.3"
- "peer0.org1.example.com:172.16.1.3"
- "peer1.org1.example.com:172.16.1.3"
启动网络
先启动node2点网络(docker-compose指令不加"-d"是为了直观看日志)
ssh node2
cd fabric-samples/first-network
docker-compose -f docker-compose-cli.yaml up
然后启动node1网络
ssh node1
cd fabric-samples/first-network
docker-compose -f docker-compose-cli.yaml up
Note:启动网络这一步花了很多时间,很多次失败,但最后是实验成功的,可以依据Create & Join Channel 后续的操作验证(只做了channel创建和配置),可看到两台机器的日志有同步操作,证明多机部署是成功的。
# **注意** 在任意一台有cli运行的节点执行(前面node1和node2的docker-compose-cli.yaml文件都配置了cli,所以都可以)
# 进入cli,后续所有指令都在这个cli中执行
docker exec -it cli bash
# 设置环境变量,后续指令要用
export CHANNEL_NAME=mychannel
# 用channel.tx创建一个channel(默认环境配置是peer0.org1.example.com)
peer channel create -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/channel.tx --tls --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
# peer0.org1.example.com 加入channel
peer channel join -b mychannel.block
# peer0.org2.example.com 加入channel(注意大写的环境变量,这是在操作peer0.org2)
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer0.org2.example.com:9051 CORE_PEER_LOCALMSPID="Org2MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt peer channel join -b mychannel.block
# 设置org1的锚节点(Org1MSPanchors.tx指定了锚节点是peer0)
peer channel update -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/Org1MSPanchors.tx --tls --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
# 设置org2的锚节点(也需要注意用peer0.org2环境变量)
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer0.org2.example.com:9051 CORE_PEER_LOCALMSPID="Org2MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt peer channel update -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/Org2MSPanchors.tx --tls --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
# peer0.org2加入channel
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer0.org2.example.com:9051 CORE_PEER_LOCALMSPID="Org2MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt peer channel join -b mychannel.block
# peer1.org2加入channel
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer1.org2.example.com:10051 CORE_PEER_LOCALMSPID="Org2MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/ca.crt peer channel join -b mychannel.block
# peer1.org1加入channel
CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer1.org1.example.com:8051 CORE_PEER_LOCALMSPID="Org1MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls/ca.crt peer channel join -b mychannel.block
lookup orderer.example.com on 127.0.0.11:53: no such host
desc = “transport: error while dialing: dial tcp 192.168.160.3:7050: connect: connection refused”
而且上边这个192.168.160.3每次运行都在变,谁能给解释下。
具体操作参照 https://hyperledger-fabric.readthedocs.io/en/release-1.4/build_network.html 的Create & join channel一节来操作,需要注意的是,文档中强调无需把所有peer都加入到channel只要把两个机构的锚节点加入channel即可,但是实际操作发现不奏效的:
Rather than join every peer, we will simply join
peer0.org2.example.com
so that we can properly update the anchor peer definitions in our channel. Since we are overriding the default environment variables baked into the CLI container, this full command will be the following:
所以需要对每个报错的peer(如本例错误信息提示的peer1.org1.example.com )添加到channel中(注意其中的四个大写配置项不要错):
$ docker exec -it cli bash
$ CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp CORE_PEER_ADDRESS=peer1.org1.example.com:8051 CORE_PEER_LOCALMSPID="Org1MSP" CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/tls/ca.crt peer channel join -b mychannel.block
(完)
公众号: 区块链架构
技术交流群: