Hyperledger Fabric 2.2实战记录(三)

八、新增orderer节点

在192.168.1.112的服务器上新增一个排序节点orderer1.example.com

在增加orderer节点时，必须保证系统通道内的大多数已加入orderer在正常工作，如果正常工作的不能达到大多数，则系统通道将彻底无法修改。

在这里遇到很多坑，尤其需要注意版本orderer（2.2版）的可执行文件必须是go1.14.6以上版本。

编辑各个主机的/etc/hosts

192.168.1.108 orderer.example.com
192.168.1.112 orderer1.example.com
192.168.1.112 peer0.org1.example.com
192.168.1.138 peer1.org1.example.com
192.168.1.111 peer0.org2.example.com
192.168.1.138 peer0.org3.example.com

1.获取orderer1.example.com的msp

mkdir ~/work/example/ca_order_server
cd ~/work/example/ca_order_server
1.初始化
fabric-ca-server init -b admin:adminpw --port 7055
2.修改fabric-ca-server-config.yaml
ca:
  # Name of this CA
  name: OrdererOrg
  # Key file (is only used to import a private key into BCCSP)
  keyfile: ../organizations/ordererOrganizations/example.com/ca/priv_sk
  # Certificate file (default: ca-cert.pem)
  certfile: ../organizations/ordererOrganizations/example.com/ca/ca.example.com-cert.pem
  # Chain file
  chainfile:
因9443端口在peer中占用，所以暂时将本配置文件中operations部分注释掉

3.启动server
fabric-ca-server start  -b admin:adminpw --port 7055

4.client登陆
mkdir ~/work/example/ca_order_client
cd ~/work/example/ca_order_client

export FABRIC_CA_CLIENT_HOME=$PWD
fabric-ca-client enroll -u http://admin:adminpw@localhost:7055

fabric-ca-client register -d --id.name orderer1.example.com --id.secret orderPW --id.type orderer -u http://0.0.0.0:7055

5.登陆orderer1.example.com获取msp
cd ~/work/example/organizations/ordererOrganizations/example.com/orderers
mkdir orderer1.example.com
cd orderer1.example.com

export FABRIC_CA_CLIENT_HOME=$PWD
fabric-ca-client enroll -u http://orderer1.example.com:[email protected]:7055 -M $FABRIC_CA_CLIENT_HOME/msp


6.声明管理员用户
mkdir msp/admincerts
cp ../../users/[email protected]/msp/signcerts/[email protected] msp/admincerts/

2.获取orderer1.example.com的tls

1.启动TLS server
mkdir ~/work/example/tlsca_order_server
cd ~/work/example/tlsca_order_server
fabric-ca-server init -b tlsadmin:tlsadminpw
2.修改配置文件
ca:
  # Name of this CA
  name: tlsca-OrdererOrg
  # Key file (is only used to import a private key into BCCSP)
  keyfile: ../organizations/ordererOrganizations/example.com/tlsca/priv_sk
  # Certificate file (default: ca-cert.pem)
  certfile: ../organizations/ordererOrganizations/example.com/tlsca/tlsca.example.com-cert.pem
  # Chain file
  chainfile:
因9443端口在peer中占用，所以暂时将本配置文件中operations部分注释掉
3.开启server
fabric-ca-server start  -b tlsadmin:tlsadminpw --port 7056
4.使用client注册账号
mkdir ~/work/example/tlsca_order_client
cd ~/work/example/tlsca_order_client
export FABRIC_CA_CLIENT_HOME=$PWD
fabric-ca-client enroll -u http://tlsadmin:tlsadminpw@localhost:7056
fabric-ca-client register -d --id.name orderer1.example.com --id.secret orderPW --id.type orderer -u http://0.0.0.0:7056
5.登录orderer1.example.com获取tls
cd ~/work/example/organizations/ordererOrganizations/example.com/orderers/orderer1.example.com

export FABRIC_CA_CLIENT_HOME=$PWD
#注意下方--csr.hosts必须加，否则在将peer加入通道时会报错
fabric-ca-client enroll -u http://orderer1.example.com:[email protected]:7056 -M $FABRIC_CA_CLIENT_HOME/tls --csr.hosts orderer1.example.com

mv tls/keystore/* tls/keystore/server.key

3.编辑系统区块

首先从peer0.org1.example.com拉取系统配置区块（在core.yaml所在目录执行）
cd ~/work/example/peer
mkdir -p conf-orderer1/sys


export CORE_PEER_MSPCONFIGPATH=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/msp/ #order组织的管理员
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/tls/ca.crt
export CORE_PEER_LOCALMSPID="OrdererMSP" #order组织mspid
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051

export ORDERER_TLSCA=${PWD}/../organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem

export CH_NAME=system-channel

peer channel fetch config conf-orderer1/sys/config_block.pb -o orderer.example.com:7050 -c $CH_NAME --tls --cafile $ORDERER_TLSCA

cd conf-orderer1/sys/
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
cp config.json modified_config.json

在modified_config.json中修改内容

位置一

找到如下位置

{
  "client_tls_cert": "ORDER TLS SERVER CERT",
  "host": "orderer.example.com",
  "port": 7050,
  "server_tls_cert": "ORDER TLS SERVER CERT"
}

其中client_tls_cert和server_tls_cert的内容是以下。

cat ~/work/example/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/tls/server.crt |base64

对于Fabric CA server产生的tls msp路径如下：

方式一：
cat ~/work/example/organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/signcerts/cert.pem |base64  > cert.txt

进入python终端获取去掉回车的证书
''.join(file('cert.txt','r').read().split('\n'))

方式二：
进入python终端
f = '/home/dev2/work/example/organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/signcerts/cert.pem'
import base64
base64.b64encode(file(f, 'r').read())

在此位置修改成如下代码（在base64转义后去掉证书中的回车）

{
  "client_tls_cert": "ORDER TLS SERVER CERT",
  "host": "orderer.example.com",
  "port": 7050,
  "server_tls_cert": "ORDER TLS SERVER CERT"
},
{
  "client_tls_cert": "ORDER1 TLS SERVER CERT",
  "host": "orderer1.example.com",
  "port": 7050,
  "server_tls_cert": "ORDER1 TLS SERVER CERT"
}

位置二

修改如下内容

"Endpoints": {
"mod_policy": "Admins",
"value": {
  "addresses": [
    "orderer.example.com:7050",
    "orderer1.example.com:7050"
  ]
},

注意如果是以下内容，则后续无法成功添加orderer，请检查orderer版本和go版本
"OrdererAddresses": {
	"mod_policy": "/Channel/Orderer/Admins",
	"value": {
	  "addresses": [
	    "orderer.example.com:7050"
	  ]
}

4.提交修改的配置区块

configtxlator proto_encode --input config.json --type common.Config > config.pb
configtxlator proto_encode --input modified_config.json --type common.Config > modified_config.pb
configtxlator compute_update --channel_id $CH_NAME --original config.pb --updated modified_config.pb --output config_update.pb
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate --output config_update.json
echo '{"payload":{"header":{"channel_header":{"channel_id":"'$CH_NAME'", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_in_envelope.json
configtxlator proto_encode --input config_update_in_envelope.json --type common.Envelope --output config_update_in_envelope.pb

cd ../../
#以OrdererMSP组织管理员的身份签名
export CORE_PEER_MSPCONFIGPATH=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/msp/ #order组织的管理员
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/tls/ca.crt
export CORE_PEER_LOCALMSPID="OrdererMSP" #order组织mspid
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
export ORDERER_TLSCA=${PWD}/../organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem

peer channel signconfigtx -f ./conf-orderer1/sys/config_update_in_envelope.pb
#提交（不需要其它管理员签名，属于OrdererOrg组织内部增加节点），
peer channel update -f ./conf-orderer1/sys/config_update_in_envelope.pb -c $CH_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_TLSCA

5.启动orderer1.example.com

#获取最新的系统通道配置区块，仍在peer0服务器上
cd ~/work/example/peer
mkdir system-genesis-block
export CORE_PEER_MSPCONFIGPATH=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/msp/ #order组织的管理员
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/tls/ca.crt
export CORE_PEER_LOCALMSPID="OrdererMSP" #order组织mspid
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
export ORDERER_TLSCA=${PWD}/../organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CH_NAME=system-channel

peer channel fetch config system-genesis-block/genesis.block -o orderer.example.com:7050 -c $CH_NAME --tls --cafile $ORDERER_TLSCA

到orderer1的服务器上，

cd ~/work/example/order

现将系统配置的最新区块复制过来

scp -r user@ip:~/work/example/peer/system-genesis-block .

对照原节点的orderer.yaml做如下修改

16 General.ListenAddress: orderer1.example.com
19 General.ListenPort: 7050
25 General.TLS.PrivateKey: ../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/keystore/server.key
27 General.TLS.Certificate: ../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/signcerts/cert.pem
29 General.TLS.RootCAs: ../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/cacerts/0-0-0-0-7056.pem
52 Cluster.ClientCertificate: ../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/signcerts/cert.pem
54 Cluster.ClientPrivateKey:
../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/keystore/server.key
89 LocalMSPDir: ../organizations/ordererOrganizations/example.com/orderers/orderer1.example.com/msp

运行orderer1

orderer start

6.将orderer1加入应用通道channel1

当前orderer1仅仅加入了系统通道，并没有加入应用通用channel1，下面将orderer1加入channel1

仍然在peer0的服务器上

cd ~/work/example/peer
mkdir -p conf-orderer1/channel1
export CORE_PEER_MSPCONFIGPATH=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/msp/ #order组织的管理员
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/tls/ca.crt
export CORE_PEER_LOCALMSPID="OrdererMSP" #order组织mspid
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
export ORDERER_TLSCA=${PWD}/../organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
export CH_NAME=channel1
#获取最新的应用通道配置区块
peer channel fetch config conf-orderer1/channel1/config_block.pb -o orderer.example.com:7050 -c $CH_NAME --tls --cafile $ORDERER_TLSCA

cd conf-orderer1/channel1
configtxlator proto_decode --input config_block.pb --type common.Block | jq .data.data[0].payload.data.config > config.json
cp config.json modified_config.json

然后按照步骤5的内容修改modified_config.json的两处内容之后

configtxlator proto_encode --input config.json --type common.Config > config.pb
configtxlator proto_encode --input modified_config.json --type common.Config > modified_config.pb
configtxlator compute_update --channel_id $CH_NAME --original config.pb --updated modified_config.pb --output config_update.pb
configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate --output config_update.json
echo '{"payload":{"header":{"channel_header":{"channel_id":"'$CH_NAME'", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_in_envelope.json
configtxlator proto_encode --input config_update_in_envelope.json --type common.Envelope --output config_update_in_envelope.pb

cd ../../
#以OrdererMSP组织管理员的身份签名
export CORE_PEER_MSPCONFIGPATH=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/msp/ #order组织的管理员
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/../organizations/ordererOrganizations/example.com/users/[email protected]/tls/ca.crt
export CORE_PEER_LOCALMSPID="OrdererMSP" #order组织mspid
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051
export ORDERER_TLSCA=${PWD}/../organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem

peer channel signconfigtx -f ./conf-orderer1/channel1/config_update_in_envelope.pb
#提交（此时不需要其它管理员签名，属于排序组织OrdererOrg内部增加节点），
peer channel update -f ./conf-orderer1/channel1/config_update_in_envelope.pb -c $CH_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_TLSCA

提交之后一段时间内会在orderer的log中看到报错，忽略即可。

orderer端
[orderer.consensus.etcdraft] logSendFailure -> ERRO 122 Failed to send StepRequest to 2, because: aborted channel=channel1 node=1

peer如果此时调用链码会看到错误
got unexpected status: SERVICE_UNAVAILABLE -- no Raft leader

等待5分钟，错误消失，即可使用orderer1来接收链码调用

九、删除orderer节点

和新增步骤类似，只是从modified_config.json中两处修改该位置中，找到对应orderer信息删除即可。

删除应用通道orderer节点和删除系统通道orderer节点两步要分开执行。

提交时，作者的环境是两个orderer同属一个排序组织。只需要一个orderer对事物变更进行签名，再提交即可。