国密Fabric-ca集群负载均衡
一、CA集群结构
Fabric-ca客户端或sdk连接到一个HA代理节点，这个HA代理节点为Fabric CA集群作负载均衡，集群中所有Fabric CA服务端共享同一个mariadb数据库。数据库用来保存用户和证书的信息。
一个服务端可能包含多个CA证书。每一个CA证书都是一个根CA证书或者一个中间CA证书。而每一个中间CA证书都有一个根CA证书或者其他的一个中间CA证书作为其父CA证书。
二、流程步骤
1、构建国密版本Fabric CA镜像
参考使用同济版本的GM Fabric CA，地址：http://github.com/tjfoc/fabric-ca-gm
进入工程地址，运行:
make fabric-ca-server
make fabric-ca-client在工程的bin目录生成可执行的fabric-ca-server和fabric-ca-client
获取到官方Fabric CA的docker镜像hyperledger/fabric-ca:x86_64-1.1.1
运行：
docker run -it hyperledger/fabric-ca /bin/bash开启新的命令行窗口运行：
docker ps -a在输出中找到刚刚运行起来的容器名称，运行：
docker inspect -f '{{.Id}}' containerName获取到容器的完整id，接着运行：
docker cp fabric-ca-server containerID:/root回到刚刚docker run的窗口
替换掉container的/usr/local/bin里面的fabric-ca-server和fabric-ca-client
exit后运行：
docker commit -m "GM" -a "awesome01" containerID hyperledger/fabric-ca:gm2、启动一个根节点
进入workspace，创建文件夹server0，进去server0创建docker-compose.yml
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
fabric-ca-server:
image: hyperledger/fabric-ca:gm
container_name: fabric-ca-server-root
ports:
- "7058:7054"
environment:
- FABRIC_CA_HOME=/etc/tjfoc/fabric-ca-server-gm
volumes:
- "./fabric-ca-server:/etc/tjfoc/fabric-ca-server-gm"
command: sh -c 'fabric-ca-server start -b admin:adminpw'创建fabric-ca-server文件夹，进入fabric-ca-server文件夹，创建fabric-ca-server-config.yaml
#############################################################################
# This is a configuration file for the fabric-ca-server command.
#
# COMMAND LINE ARGUMENTS AND ENVIRONMENT VARIABLES
# ------------------------------------------------
# Each configuration element can be overridden via command line
# arguments or environment variables. The precedence for determining
# the value of each element is as follows:
# 1) command line argument
# Examples:
# a) --port 443
# To set the listening port
# b) --ca-keyfile ../mykey.pem
# To set the "keyfile" element in the "ca" section below;
# note the '-' separator character.
# 2) environment variable
# Examples:
# a) FABRIC_CA_SERVER_PORT=443
# To set the listening port
# b) FABRIC_CA_SERVER_CA_KEYFILE="../mykey.pem"
# To set the "keyfile" element in the "ca" section below;
# note the '_' separator character.
# 3) configuration file
# 4) default value (if there is one)
# All default values are shown beside each element below.
#
# FILE NAME ELEMENTS
# ------------------
# The value of all fields whose name ends with "file" or "files" are
# name or names of other files.
# For example, see "tls.certfile" and "tls.clientauth.certfiles".
# The value of each of these fields can be a simple filename, a
# relative path, or an absolute path. If the value is not an
# absolute path, it is interpretted as being relative to the location
# of this configuration file.
#
#############################################################################
# Server's listening port (default: 7054)
port: 7054
# Enables debug logging (default: false)
debug: true
# Size limit of an acceptable CRL in bytes (default: 512000)
crlsizelimit: 512000
#############################################################################
# TLS section for the server's listening port
#
# The following types are supported for client authentication: NoClientCert,
# RequestClientCert, RequireAnyClientCert, VerifyClientCertIfGiven,
# and RequireAndVerifyClientCert.
#
# Certfiles is a list of root certificate authorities that the server uses
# when verifying client certificates.
#############################################################################
tls:
# Enable TLS (default: false)
enabled: false
# TLS for the server's listening port
certfile: ca-cert.pem
keyfile: ca-key.pem
clientauth:
type: noclientcert
certfiles:
#############################################################################
# The CA section contains information related to the Certificate Authority
# including the name of the CA, which should be unique for all members
# of a blockchain network. It also includes the key and certificate files
# used when issuing enrollment certificates (ECerts) and transaction
# certificates (TCerts).
# The chainfile (if it exists) contains the certificate chain which
# should be trusted for this CA, where the 1st in the chain is always the
# root CA certificate.
#############################################################################
ca:
# Name of this CA
name:
# Key file (default: ca-key.pem)
keyfile: ca-key.pem
# Certificate file (default: ca-cert.pem)
certfile: ca-cert.pem
# Chain file (default: chain-cert.pem)
chainfile: ca-chain.pem
#############################################################################
# The registry section controls how the fabric-ca-server does two things:
# 1) authenticates enrollment requests which contain a username and password
# (also known as an enrollment ID and secret).
# 2) once authenticated, retrieves the identity's attribute names and
# values which the fabric-ca-server optionally puts into TCerts
# which it issues for transacting on the Hyperledger Fabric blockchain.
# These attributes are useful for making access control decisions in
# chaincode.
# There are two main configuration options:
# 1) The fabric-ca-server is the registry.
# This is true if "ldap.enabled" in the ldap section below is false.
# 2) An LDAP server is the registry, in which case the fabric-ca-server
# calls the LDAP server to perform these tasks.
# This is true if "ldap.enabled" in the ldap section below is true,
# which means this "registry" section is ignored.
#############################################################################
registry:
# Maximum number of times a password/secret can be reused for enrollment
# (default: -1, which means there is no limit)
maxenrollments: -1
# Contains identity information which is used when LDAP is disabled
identities:
- name: admin
pass: adminpw
type: client
affiliation: ""
maxenrollments: -1
attrs:
hf.Registrar.Roles: "client,user,peer,validator,auditor"
hf.Registrar.DelegateRoles: "client,user,validator,auditor"
hf.Revoker: true
hf.IntermediateCA: true
#############################################################################
# Database section
# Supported types are: "sqlite3", "postgres", and "mysql".
# The datasource value depends on the type.
# If the type is "sqlite3", the datasource value is a file name to use
# as the database store. Since "sqlite3" is an embedded database, it
# may not be used if you want to run the fabric-ca-server in a cluster.
# To run the fabric-ca-server in a cluster, you must choose "postgres"
# or "mysql".
#############################################################################
db:
type: mysql
datasource: root:hello123@tcp(192.168.0.5:3306)/fabric_ca_root?parseTime=true
tls:
enabled: false
certfiles:
- db-server-cert.pem
client:
certfile: db-client-cert.pem
keyfile: db-client-key.pem
#############################################################################
# LDAP section
# If LDAP is enabled, the fabric-ca-server calls LDAP to:
# 1) authenticate enrollment ID and secret (i.e. username and password)
# for enrollment requests;
# 2) To retrieve identity attributes
#############################################################################
ldap:
# Enables or disables the LDAP client (default: false)
# If this is set to true, the "registry" section is ignored.
enabled: false
# The URL of the LDAP server
url: ldap://:@:/
# Automatically generate non-default CAs. The names of these
# additional CAs are "ca1", "ca2", ... "caN", where "N" is
# This is particularly useful in a development environment to quickly set up
# multiple CAs.
#
# 2) --cafiles
# For each CA config file in the list, generate a separate signing CA. Each CA
# config file in this list MAY contain all of the same elements as are found in
# the server config file except port, debug, and tls sections.
#
# Examples:
# fabric-ca-server start -b admin:adminpw --cacount 2
#
# fabric-ca-server start -b admin:adminpw --cafiles ca/ca1/fabric-ca-server-config.yaml
# --cafiles ca/ca2/fabric-ca-server-config.yaml
#
#############################################################################
cacount:
cafiles:
#############################################################################
# Intermediate CA section
#
# The relationship between servers and CAs is as follows:
# 1) A single server process may contain or function as one or more CAs.
# This is configured by the "Multi CA section" above.
# 2) Each CA is either a root CA or an intermediate CA.
# 3) Each intermediate CA has a parent CA which is either a root CA or another intermediate CA.
#
# This section pertains to configuration of #2 and #3.
# If the "intermediate.parentserver.url" property is set,
# then this is an intermediate CA with the specified parent
# CA.
#
# parentserver section
# url - The URL of the parent server
# caname - Name of the CA to enroll within the server
#
# enrollment section used to enroll intermediate CA with parent CA
# profile - Name of the signing profile to use in issuing the certificate
# label - Label to use in HSM operations
#
# tls section for secure socket connection
# certfiles - PEM-encoded list of trusted root certificate files
# client:
# certfile - PEM-encoded certificate file for when client authentication
# is enabled on server
# keyfile - PEM-encoded key file for when client authentication
# is enabled on server
#############################################################################
intermediate:
parentserver:
url:
caname:
enrollment:
hosts:
profile:
label:
tls:
certfiles:
client:
certfile:
keyfile: 回到目录server0，启动根节点：
docker-compose up -d运行以下命令注册一个具有”hf.IntermediateCA=true“属性的admin1账号：
fabric-ca-client enroll -u http://admin:adminpw@localhost:7058
fabric-ca-client register -u http://admin:adminpw@localhost:7058 --id.name admin1 --id.secret adminpw --id.affiliation org1 --id.type client --id.attrs 'hf.IntermediateCA=true,hf.Revoker=true,admin=true:ecert'2、创建Fabric CA集群
进入workspace，创建server、server1、server2文件夹
进入server
创建docker-compose.yaml
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
fabric-ca-server:
image: hyperledger/fabric-ca:gm
container_name: fabric-ca-server1
ports:
- "8054:7054"
environment:
- FABRIC_CA_HOME=/etc/tjfoc/fabric-ca-server-gm
volumes:
- "../fabric-ca-server:/etc/tjfoc/fabric-ca-server-gm"
command: sh -c 'fabric-ca-server start -b admin:adminpw -u http://admin1:[email protected]:7058'这里使用上一步中创建的根CA及admin1账号，填入-u参数里作为父CA。
回到workspace，server1和server2的步骤与上面一样，创建docker-compose.yaml，区别是修改container_name，分别是fabric-ca-server2，fabric-ca-server3
在workspace创建fabric-ca-server文件夹，进入文件夹创建fabric-ca-server-config.yaml
与根节点大体上相同，修改：db中datasource中的数据库，改为fabric_ca
回到workspace，创建startCAs.sh
cd server
docker-compose up -d
cd ../server1
docker-compose up -d
cd ../server2
docker=compose up -d运行startCAs.sh脚本，可以看到docker容器列表中，CA集群已经开始运行，分别占用端口8054，9054，10054
3、使用haproxy实现高可用和负载均衡
yum install -y haproxy修改/etc/haproxy/haproxy.cfg，添加：
listen fabric-ca 0.0.0.0:7054
mode tcp
option tcplog
balance roundrobin
server ca1 127.0.0.1:8054 check inter 5000 rise 2 fall 2
server ca2 127.0.0.1:9054 check inter 5000 rise 2 fall 2
server ca3 127.0.0.1:10054 check inter 5000 rise 2 fall 2运行：
service haproxy restart通过lsof -i:7054可以看到haproxy已运行。
最终目录结构为：
.
├── fabric-ca-server
│ ├── ca-cert.pem
│ ├── ca-chain.pem
│ ├── fabric-ca-server-config.yaml
│ └── msp
│ ├── cacerts
│ ├── keystore
│ │ └── b525d57d09f3a4440d34e1ce0e54bf969592835004842260634b99a0a52c169e_sk
│ └── signcerts
├── README.md
├── server
│ ├── docker-compose.yml
│ └── fabric-ca-server
│ ├── ca-cert.pem
│ ├── ca-chain.pem
│ ├── fabric-ca-server-config.yaml
│ └── msp
│ ├── cacerts
│ ├── keystore
│ │ └── bd057bbcd30f18effd7abc810d3f74daba62ac285afb0aaf5c729f6d4faae8dc_sk
│ └── signcerts
├── server0
│ ├── docker-compose.yml
│ └── fabric-ca-server
│ ├── ca-cert.pem
│ ├── fabric-ca-server-config.yaml
│ └── msp
│ └── keystore
│ └── 01fff9a0099e1e2c9411966b8ee4e70651976af1e54b60a12895cd928a30fa75_sk
├── server1
│ ├── docker-compose.yml
│ └── fabric-ca-server
│ ├── ca-cert.pem
│ ├── ca-chain.pem
│ ├── fabric-ca-server-config.yaml
│ └── msp
│ ├── cacerts
│ ├── keystore
│ │ └── 2a74bc776a1158dc97e6febf4073f8cf821d0e9af8c6d5ff89000fe7b3c3e911_sk
│ └── signcerts
├── server2
│ ├── docker-compose.yml
│ └── fabric-ca-server
│ ├── ca-cert.pem
│ ├── ca-chain.pem
│ ├── fabric-ca-server-config.yaml
│ └── msp
│ ├── cacerts
│ ├── keystore
│ │ └── 36b92dee1e83ddc5b79b37bb6f18bc7856ca5bf36c52330dd8d0ad06aa8aaf27_sk
│ └── signcerts
├── startServers.sh
└── stopServers.sh
三、测试
运行：
fabric-ca-client enroll -u http://admin:[email protected]:7054
fabric-ca-client register --id.name admin1 --id.secret adminpw --id.affiliation org1.department1 --id.type user --id.attrs 'hf.Revoker=true,admin=true:ecert'如果不报错说明已搭建成功
四、错误处理
1、Authorization failure10
数据库的datasource没有加上
?parseTime=true2、Authorization failure9
Fabric CA集群没有使用统一的映射文件夹