前言
在Hyperledger Fabric无系统通道启动及通道的创建和删除中,我们已经完成了以无系统通道的方式启动 Hyperledger Fabric 网络,并将链码安装到指定通道。但目前为止,实验中的 orderer 服务都是通过单独的排序组织来维护且只有一个,那能不能不要排序组织而使用普通组织来运行维护多个 orderer 服务呢?当然是可以的,本文将在之前的实验基础上,启动一个没有 orderer 组织的 Fabric 网络,网络中包含三个组织且每个组织运行维护着一个 Raft 协议的 orderer 节点,最后成功在其上部署运行链码。
背景介绍
实验准备
本文网络结构直接将 Hyperledger Fabric无系统通道启动及通道的创建和删除 中创建的 3_RunWithNoSystemChannel
复制为 4-1_RunOrdererByOneself
(建议直接将本案例仓库 FabricLearn 下的 4-1_RunOrdererByOneself
目录拷贝到本地运行)。默认情况下,所有命令皆在 4-1_RunOrdererByOneself
根目录下执行,在开始后面的实验前按照以下命令启动基础实验网络(主要修改为删除 orderer 组织相关配置):
- 设置环境变量
source envpeer1soft
- 启动CA网络
./0_Restart.sh
本文工作
以无排序组织的方式启动 Hyperledger Fabric 网络,其中包含三个组织—— soft 、 web 、 hard , 每个组织都运行维护着一个 peer 节点和一个 orderer,并使用 osnadmin
工具通过 orderer
的 admin
服务使 orderer
加入这两条通道(实验代码已上传至:https://github.com/wefantasy/FabricLearn 的 4-1_RunOrdererByOneself
下):
项 | 运行端口 | 说明 |
---|---|---|
council.ifantasy.net |
7050 | council 组织的 CA 服务, 为联盟链网络提供 TLS-CA 服务 |
soft.ifantasy.net |
7250 | soft 组织的 CA 服务, 包含成员: peer1 、 admin1 |
peer1.soft.ifantasy.net |
7251 | soft 组织的 peer1 成员节点 |
orderer1.soft.ifantasy.net |
8251 | soft 组织的 orderer1 服务 |
orderer1.soft.ifantasy.net |
8252 | soft 组织的 orderer1 服务的 admin 服务 |
web.ifantasy.net |
7350 | web 组织的 CA 服务, 包含成员: peer1 、 admin1 |
peer1.web.ifantasy.net |
7351 | web 组织的 peer1 成员节点 |
orderer1.soft.ifantasy.net |
8351 | web 组织的 orderer1 服务 |
orderer1.soft.ifantasy.net |
8352 | web 组织的 orderer1 服务的 admin 服务 |
hard.ifantasy.net |
7450 | hard 组织的 CA 服务, 包含成员: peer1 、 admin1 |
peer1.hard.ifantasy.net |
7451 | hard 组织的 peer1 成员节点 |
orderer1.soft.ifantasy.net |
8451 | hard 组织的 orderer1 服务 |
orderer1.soft.ifantasy.net |
8452 | hard 组织的 orderer1 服务的 admin 服务 |
实验步骤
配置文件
修改配置文件
compose/docker-compose.yaml
,删除所有关于 orderer 组织的配置,并新增 hard 组织相关容器和普通组织的 orderer 容器:hard.ifantasy.net: container_name: hard.ifantasy.net extends: file: docker-base.yaml service: ca-base command: sh -c 'fabric-ca-server start -d -b ca-admin:ca-adminpw --port 7050' environment: - FABRIC_CA_SERVER_CSR_CN=hard.ifantasy.net - FABRIC_CA_SERVER_CSR_HOSTS=hard.ifantasy.net volumes: - ${LOCAL_CA_PATH}/hard.ifantasy.net/ca:${DOCKER_CA_PATH}/ca ports: - 7450:7050 peer1.hard.ifantasy.net: container_name: peer1.hard.ifantasy.net extends: file: docker-base.yaml service: peer-base environment: - CORE_PEER_ID=peer1.hard.ifantasy.net - CORE_PEER_LISTENADDRESS=0.0.0.0:7051 - CORE_PEER_ADDRESS=peer1.hard.ifantasy.net:7051 - CORE_PEER_LOCALMSPID=hardMSP - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.hard.ifantasy.net:7051 volumes: - ${LOCAL_CA_PATH}/hard.ifantasy.net/registers/peer1:${DOCKER_CA_PATH}/peer ports: - 7451:7051 orderer1.soft.ifantasy.net: container_name: orderer1.soft.ifantasy.net extends: file: docker-base.yaml service: orderer-base environment: - ORDERER_HOST=orderer1.soft.ifantasy.net - ORDERER_GENERAL_LOCALMSPID=softMSP - ORDERER_GENERAL_LISTENPORT=8251 volumes: - ${LOCAL_CA_PATH}/soft.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer ports: - 8251:8251 - 8252:8888 - 8253:9999 orderer1.web.ifantasy.net: container_name: orderer1.web.ifantasy.net extends: file: docker-base.yaml service: orderer-base environment: - ORDERER_HOST=orderer1.web.ifantasy.net - ORDERER_GENERAL_LOCALMSPID=webMSP - ORDERER_GENERAL_LISTENPORT=8351 volumes: - ${LOCAL_CA_PATH}/web.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer ports: - 8351:8351 - 8352:8888 - 8353:9999 orderer1.hard.ifantasy.net: container_name: orderer1.hard.ifantasy.net extends: file: docker-base.yaml service: orderer-base environment: - ORDERER_HOST=orderer1.hard.ifantasy.net - ORDERER_GENERAL_LOCALMSPID=hardMSP - ORDERER_GENERAL_LISTENPORT=8451 volumes: - ${LOCAL_CA_PATH}/hard.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer ports: - 8451:8451 - 8452:8888 - 8453:9999
修改配置文件
config/configtx.yaml
,源文件太长在此不贴,其主要修改内容为:各组织的环境变量文件中添加 orderer 服务的管理证书环境变量,以
envpeer1soft
为例:export ORDERER_CA=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/tlscacerts/tls-council-ifantasy-net-7050.pem export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/signcerts/cert.pem export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
将
envpeer1soft
复制为envpeer1hard
作为 hard 组织的环境变量,其内容为:export LOCAL_ROOT_PATH=$PWD export LOCAL_CA_PATH=$LOCAL_ROOT_PATH/orgs export DOCKER_CA_PATH=/tmp export COMPOSE_PROJECT_NAME=fabriclearn export DOCKER_NETWORKS=network export FABRIC_BASE_VERSION=2.4 export FABRIC_CA_VERSION=1.5 echo "init terminal hard" export FABRIC_CFG_PATH=$LOCAL_ROOT_PATH/config export CORE_PEER_TLS_ENABLED=true export CORE_PEER_LOCALMSPID="hardMSP" export CORE_PEER_ADDRESS=peer1.hard.ifantasy.net:7451 export CORE_PEER_TLS_ROOTCERT_FILE=$LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem export CORE_PEER_MSPCONFIGPATH=$LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp export ORDERER_CA=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/tlscacerts/tls-council-ifantasy-net-7050.pem export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/signcerts/cert.pem export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/hard.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
注册用户
在注册脚本 1_RegisterUser.sh
中删除 orderer 组织账户并添加三组织排序服务的 msp 账户和 tls-msp 账户,默认密码与账户名相同:
登录账户
在登录脚本 2_EnrollUser.sh
中删除 orderer 组织相关内容并添加三组织排序服务的 msp 账户和 tls-msp 账户,如 soft 组织下新增登录 orderer1 的代码:
echo "Enroll Orderer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer1:[email protected]:7250
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1soft:[email protected]:7050 --enrollment.profile tls --csr.hosts orderer1.soft.ifantasy.net
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/orderer1/msp/admincerts/cert.pem
创建通道并加入
在执行上述脚本后,在脚本 3_CreateChannel.sh
中创建通道并使所有节点加入:
启动 peer 和 orderer 容器:
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d peer1.soft.ifantasy.net peer1.web.ifantasy.net peer1.hard.ifantasy.net docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d orderer1.soft.ifantasy.net orderer1.web.ifantasy.net orderer1.hard.ifantasy.net
创建通道文件
testchannel
:configtxgen -profile OrgsChannel -outputCreateChannelTx $LOCAL_ROOT_PATH/data/testchannel.tx -channelID testchannel configtxgen -profile OrgsChannel -outputBlock $LOCAL_ROOT_PATH/data/testchannel.block -channelID testchannel
soft 组织的 orderer 服务加入通道:
source envpeer1soft osnadmin channel list -o orderer1.soft.ifantasy.net:8252 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY osnadmin channel join -o orderer1.soft.ifantasy.net:8252 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY" osnadmin channel list -o orderer1.soft.ifantasy.net:8252 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
web 组织的 orderer 服务加入通道:
source envpeer1web osnadmin channel list -o orderer1.web.ifantasy.net:8352 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY osnadmin channel join -o orderer1.web.ifantasy.net:8352 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY" osnadmin channel list -o orderer1.web.ifantasy.net:8352 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
hard 组织的 orderer 服务加入通道:
source envpeer1hard osnadmin channel list -o orderer1.hard.ifantasy.net:8452 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY osnadmin channel join -o orderer1.hard.ifantasy.net:8452 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY" osnadmin channel list -o orderer1.hard.ifantasy.net:8452 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
将通道文件复制到各组织资产目录下:
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/soft.ifantasy.net/assets/ cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/web.ifantasy.net/assets/ cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/hard.ifantasy.net/assets/
各组织 peer 节点加入通道:
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/soft.ifantasy.net/assets/ cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/web.ifantasy.net/assets/ cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/hard.ifantasy.net/assets/
部署测试链码
在执行上述脚本后,在脚本 4_TestChaincode.sh
中安装链码并调用执行:
各组织安装测试链码:
source envpeer1soft # peer lifecycle chaincode package basic.tar.gz --path asset-transfer-basic/chaincode-go --label basic_1 peer lifecycle chaincode install basic.tar.gz peer lifecycle chaincode queryinstalled source envpeer1web peer lifecycle chaincode install basic.tar.gz peer lifecycle chaincode queryinstalled source envpeer1hard peer lifecycle chaincode install basic.tar.gz peer lifecycle chaincode queryinstalled
设置链码 ID 环境变量:
export CHAINCODE_ID=basic_1:06613e463ef6694805dd896ca79634a2de36fdf019fa7976467e6e632104d718
soft 组织批准链码:
source envpeer1soft peer lifecycle chaincode approveformyorg -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
web 组织批准链码:
source envpeer1web peer lifecycle chaincode approveformyorg -o orderer1.web.ifantasy.net:8351 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
hard 组织批准链码:
source envpeer1hard peer lifecycle chaincode approveformyorg -o orderer1.hard.ifantasy.net:8451 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
检查链码批准情况:
peer lifecycle chaincode checkcommitreadiness -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --init-required
测试调用链码:
source envpeer1soft peer lifecycle chaincode commit -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --init-required --version 1.0 --sequence 1 --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE peer lifecycle chaincode querycommitted --channelID testchannel --name basic -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE peer chaincode invoke --isInit -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["InitLedger"]}' sleep 3 peer chaincode invoke -o orderer1.soft.ifantasy.net:8251 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["GetAllAssets"]}'
常见错误
没有领导节点
Error: failed to send transaction: got unexpected status: SERVICE_UNAVAILABLE -- no Raft leader
上述错误归结起来就是 orderer 之间没有选出领导节点,此时应该检查:
- 网络中 orderer 节点的数量是否为
2n+1
个,否则可能无法完成选举 - 各 orderer 容器的
ORDERER_GENERAL_LOCALMSPID
配置是否正确,必须为自身所属组织的 MSPID - 检查
configtx.yaml
中各组织的Policies
配置是否正确 排序节点之间无法通信
2022-04-09 05:32:07.086 UTC 0032 ERRO [orderer.consensus.etcdraft] logSendFailure -> Failed to send StepRequest to 3, because: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 172.19.0.10:8451: connect: connection refused" channel=syschannel node=1
上述错误的原因是 orderer 节点间无法通信, 此时应该检查
configtx.yaml
中相关的 orderer 地址是否正确。这里有个大坑:所有configtx.yaml
文件内的 orderer 相关配置的端口必须设置为容器内ORDERER_GENERAL_LISTENPORT
的监听端口,而不是容器外的映射端口,假如 orderer 容器配置如下图,configtx.yaml
中的 orderer 端口必须为7050
而不能填8251
(所以为了避免冲突,强烈建议这两个端口设置成一样的 8251)。peer 节点之间无法通信
Error: timed out waiting for txid on all peers 2022-04-10 02:57:37.135 UTC 00a1 WARN [peer.blocksprovider] DeliverBlocks -> Got error while attempting to receive blocks: block from orderer could not be verified: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Writers' sub-policies to be satisfied channel=testchannel orderer-address=orderer1.soft.ifantasy.net:8251
错误原因是没有操作权限,通常是
configtx.yaml
中的策略问题,在本实验中如果三个组织的Policies
都设置为下列内容则会触发本错误:Policies: Readers: Type: Signature Rule: "OR('softMSP.admin', 'softMSP.peer', 'softMSP.client')" Writers: Type: Signature Rule: "OR('softMSP.admin', 'softMSP.client')" Admins: Type: Signature Rule: "OR('softMSP.admin')" Endorsement: Type: Signature Rule: "OR('softMSP.peer')"
此时需要将任意组织(比如 web)的
Readers
和Writers
的Rule
改为menber
即可解决,解决后实验各步骤结果符合预期:Policies: Readers: Type: Signature Rule: "OR('webMSP.member')" Writers: Type: Signature Rule: "OR('webMSP.member')" Admins: Type: Signature Rule: "OR('webMSP.admin')" Endorsement: Type: Signature Rule: "OR('webMSP.peer')"
至于为什么会导致如此尚未发现,猜测是普通组织的策略与排序节点所需要的策略存在冲突,因此建议排序服务独立于普通组织。