bigchaindb集群部署（5节点）
在安装bigchainDB集群之前，我们需要了解bigchainDB的一些相关术语：
- bigchainDB节点：是一台或一群相近的节点的集合。bigchainDB节点上运行了mongoDB server、bigchainDB server及一些相关的软件
- bigchainDB集群：一群互联的bigchainDB节点组成了bigchainDB集群。集群中的节点运行相同的软件。一个集群可能需要有其他的额外的节点来完成集群监控
- bigchainDB联盟（Consortium）：一个集群中管理节点的组织或个人属于bigchainDB联盟。当集群由一个公司管理运行，则联盟为该公司
bigchainDB的节点按照使用目的可以分为dev/test节点（开发与测试）、bare-bones节点（部署在云上）或者production节点（拥有更多的组件）。一个production节点必须包括BigchainDB Server、MongoDB Server 3.4+ (mongod)以及Scalable storage for MongoDB。这些关键组件的关系如下图所示。其中BigChainDB server必须能够与所有mongod实例相连，所有的mongod也相互连接。
安装环境
| hostname | ip | os |
|---|---|---|
| node-admin | 10.0.0.70 | ubuntu 14.04 desktop |
| node1 | 10.0.0.71 | ubuntu 14.04 server |
| node2 | 10.0.0.72 | ubuntu 14.04 server |
| node3 | 10.0.0.73 | ubuntu 14.04 server |
| node4 | 10.0.0.74 | ubuntu 14.04 server |
| node5 | 10.0.0.75 | ubuntu 14.04 server |
其中node-admin用来使用ansible来执行远程命令（为了部署便利），node1-node5为bigchain DB节点。所有机器用户均为root
为了使用ansible，我们首先在所有节点上安装openssh-server，并允许root用户ssh。具体方式为修改/etc/ssh/sshd_config，修改为PermitRootLogin yes，然后重启ssh服务。
安装与配置ansible
接下来在node-admin上安装与配置ansible。
apt-get update && apt-get upgrade
apt-get install ansible修改/etc/ansible/hosts:
root@clean:~# grep -Ev "^$|#" /etc/ansible/hosts
[bigchain]
10.0.0.71 ansible_ssh_pass=123
10.0.0.72 ansible_ssh_pass=123
10.0.0.73 ansible_ssh_pass=123
10.0.0.74 ansible_ssh_pass=123
10.0.0.75 ansible_ssh_pass=123
root@clean:~#设置为首次链接不需要key认证
sed -i "s/^#\(host_key_checking\).*/\1 = False/g" /etc/ansible/ansible.cfg测试：ansible bigchain -m ping
安装bigchainDB
以下所有命令默认在node-admin上执行
- 同步时钟
# 测试是否能ping通cn.pool.ntp.org
ansible bigchain -m command -a "ping cn.pool.ntp.org -c 4"
# 同步
ansible bigchain -m command -a "ntpdate cn.pool.ntp.org"- 安装mongoDB
编辑sources.list
echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list备用 清华的源
echo "deb [ arch=amd64 ] https://mirrors.tuna.tsinghua.edu.cn/mongodb/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list编辑mongod.yml，内容如下：
---
- hosts: bigchain
remote_user: root
# invoke setup module to gather facts before executing tasks
gather_facts: true
tasks:
- name: debug
debug: msg="myhostname={{ansible_hostname}}"
- name: apt-key
command: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
- name: sources.list
template:
src: mongodb-org-3.4.list
dest: /etc/apt/sources.list.d/mongodb-org-3.4.list
owner: root
group: root
mode: 0644
- name: update
command: apt-get update -y
- name: install packages
apt:
name: "{{item}}"
force: yes
with_items:
- mongodb-org
- g++
- python3-dev
- libffi-dev
- python3-pip
- name: setuptools
command: pip3 install --upgrade pip setuptools
- name: db
command: mkdir -p /data/db运行yml(耗时很长，因为需要完成apt-get update与安装许多包)
ansible-playbook mongod.yml- 启动mongod
# 启动mongo
ansible bigchain -m command -a "mongod --replSet bigchain --fork --logpath /var/log/mongodb/mongod.log"若出现address已经被绑定这种错误的话，可以在错误节点上执行
kill -9 $(ps -ef | grep mongod | awk 'NR>1{print p}{p=$2}')
# kill全部
ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep mongod | awk 'NR>1{print p}{p=\$2}')"若mongod出现错误：
2017-06-21T09:56:34.552+0800 W - [initandlisten] Detected unclean shutdown - /var/lib/mongodb/mongod.lock is not empty.
2017-06-21T09:56:34.568+0800 E NETWORK [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2017-06-21T09:56:34.568+0800 I - [initandlisten] Fatal Assertion 28578 at src/mongo/util/net/listen.cpp 195可以在错误节点上执行：
rm /data/db/mongod.lock
mongod --repair
ansible bigchain -m raw -a "rm /data/db/mongd.lock"
ansible bigchain -m raw -a "mongod --repair"- 创建副本集
在node1上执行
mongo
config = {_id: 'bigchain', members: [{
"_id": 0,
"host":"10.0.0.71:27017"
}]
}
rs.initiate(config);
rs.add("10.0.0.72:27017")
rs.add("10.0.0.73:27017")
rs.add("10.0.0.74:27017")
rs.add("10.0.0.75:27017")- 安装bigchainDB
ansible bigchain -m command -a "pip3 install --upgrade bigchaindb"
ansible bigchain -m command -a "bigchaindb -y configure mongodb"- 修改bigchainDB配置
允许接收所有地址信息
ansible bigchain -m raw -a "sed -i 's/\(\"bind\": \"\)localhost:9984\"/\10.0.0.0:9984\"/g' /root/.bigchaindb"修改replSet名称为bigchain（之前mongod的replset名称）：
ansible bigchain -m raw -a "sed -i 's/\(\"replicaset\": \"\).*/\1bigchain\",/g' /root/.bigchaindb"还需要修改keyring选项，使之存储除本节点之外的所有其他的节点的keyring的公钥。先来获取所有节点的keyring的公钥
ansible bigchain -m raw -a "cat .bigchaindb | grep public | awk -F\\\" '{print \$4}'"输出如下：
10.0.0.71 | success | rc=0 >>
7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ
10.0.0.72 | success | rc=0 >>
GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv
10.0.0.74 | success | rc=0 >>
Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv
10.0.0.73 | success | rc=0 >>
8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv
10.0.0.75 | success | rc=0 >>
GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH我们将这些信息写入一个脚本(conf.py)里
import sys
import json
keyring = {
"10.0.0.71": "7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ",
"10.0.0.72": "GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv",
"10.0.0.74": "Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv",
"10.0.0.73": "8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv",
"10.0.0.75": "GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH"
}
rets = []
for key, value in keyring.items():
if key != sys.argv[1]:
rets.append(value)
conf = json.load(open("/root/.bigchaindb"))
conf['keyring'] = rets
json.dump(conf, open("/root/.bigchaindb", "w"), indent=2)编辑bigchain.yml，用来分发该脚本，并执行该脚本
---
- hosts: bigchain
remote_user: root
# invoke setup module to gather facts before executing tasks
gather_facts: true
tasks:
- name: debug
debug: msg="my ip of eth0 is {{ansible_eth0.ipv4.address}}"
- name: copy file
template:
src: conf.py
dest: /root/conf.py
owner: root
group: root
mode: 0644
- name: modify configuration
command: python conf.py {{ansible_eth0.ipv4.address}}执行：
ansible-playbook bigchain.yml执行成功后，可以看到node1-node5的.bigchaindb里keyring均成功写入。
- 启动bigchainDB
任选一节点（如node1上）执行
bigchaindb init然后启动所有节点的bigchaindb
bigchaindb start
# 或者后台启动
nohup bigchaindb start > /dev/null 2>&1 &
# 启动全部
ansible bigchain -m shell -a "nohup bigchaindb start > /dev/null 2>&1 &"
# kill全部
ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep bigchaindb | awk 'NR>1{print p}{p=\$2}')"使用bigchaindb
首先得安装bigchaindb的python driver
ansible bigchain -m command -a "apt-get install git -y"
ansible bigchain -m command -a "apt-get install libssl-dev -y"
ansible bigchain -m command -a "pip3 install --process-dependency-links git+https://github.com/bigchaindb/bigchaindb-driver.git"官网上的测试用例：https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html
# coding=utf-8
# author: Wu Luo
import bigchaindb_driver
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
from time import sleep
import json
from sys import exit
import sys
class BigChainAPI(object):
def __init__(self, host, port=9984, conf='/root/.bigchaindb', timeout=60):
'''
:param host: cluster host of bigchaindb
:param port: port of bigchaindb
:param conf: the configuration file of bigchaindb
:param timeout: waiting vote for `timeout` seconds
:return:
'''
self.host = host
self.port = int(port)
self.bdb = BigchainDB("http://%s:%d" % (self.host, self.port))
self.conf = conf
self.timeout = int(timeout)
self.loadUserKey()
def loadUserKey(self):
'''
load user key from configuration files, as the valid keypairs are
designated in our deployment
:return:
'''
configuration = json.load(open(self.conf, "r"))
self.user = configuration['keypair']
def waitVote(self, txid):
'''
wait the transaction to be voted as valid
:param txid: the transaction id
:return: True/False
'''
print("txid is ", txid)
trials = 0
while trials < self.timeout:
try:
if self.bdb.transactions.status(txid).get('status') == 'valid':
print('Tx valid in:', trials, 'secs')
break
elif self.bdb.transactions.status(txid).get('status') == 'invalid':
print('Tx invalid in:', trials, 'secs')
print(self.bdb.transactions.status(txid))
break
else:
trials += 1
print("trials " + str(trials))
sleep(1)
except bigchaindb_driver.exceptions.NotFoundError:
trials += 1
sleep(1)
if trials == self.timeout:
print('Tx is still being processed... Bye!')
return False
return True
def writeTransaction(self, data, metadata=None):
'''
write data into bigchaindb
:param data: the data to write. it should be a dict
:param metadata: the metadata to write. it should be a dict
:return: the transaction id
'''
_asset = {
'data': data
}
prepared_creation_tx = self.bdb.transactions.prepare(
operation='CREATE',
signers=self.user['public'],
asset=_asset,
metadata=metadata
)
fulfilled_creation_tx = self.bdb.transactions.fulfill(
prepared_creation_tx,
private_keys=self.user['private']
)
sent_creation_tx = self.bdb.transactions.send(fulfilled_creation_tx)
txid = fulfilled_creation_tx['id']
if not self.waitVote(txid):
return False
return txid
def transferAsset(self, asset_id, publicKeyOfTargetUser):
'''
transfer an asset to another user
:param asset_id: the asset to transfer
:param publicKeyOfTargetUser: the public key of the target user
It should be generated by generate_keypair()
:return: the transaction id
'''
transfer_asset = {
'id': asset_id
}
# query the transaction which contains the asset
data = self.bdb.transactions.retrieve(asset_id)
output_index = 0
output = data['outputs'][output_index]
transfer_input = {
'fulfillment': output['condition']['details'],
'fulfills': {
'output': output_index,
# in other version of BigchainDB, the key should be 'txid'
'transaction_id': data['id']
},
'owners_before': output['public_keys']
}
prepared_transfer_tx = self.bdb.transactions.prepare(
operation='TRANSFER',
asset=transfer_asset,
inputs=transfer_input,
recipients=publicKeyOfTargetUser,
)
fulfilled_transfer_tx = self.bdb.transactions.fulfill(
prepared_transfer_tx,
private_keys=self.user['private'],
)
sent_transfer_tx = self.bdb.transactions.send(fulfilled_transfer_tx)
txid = fulfilled_transfer_tx['id']
if not self.waitVote(txid):
return False
return txid
def queryTransaction(self, txid):
'''
retrieve a transaction
:param txid: the txid of the transaction to query
:return: the data of the retrieved transaction
'''
data = self.bdb.transactions.retrieve(txid)
return data['asset']
def test():
bigchainAPI = BigChainAPI(host='10.0.0.71', port=9984, timeout=300)
# write an transaction
data = {
'author': 'Wu Luo'
}
txid = bigchainAPI.writeTransaction(data)
if txid == False:
print(">>> waiting for vote")
return False
# query this transaction
query_data = bigchainAPI.queryTransaction(txid)
print(">>> txid is " + txid)
print(">>> data retrieved from bigchainDB")
print(query_data)
# transfer
user = generate_keypair()
newid = bigchainAPI.transferAsset(txid, user.public_key)
print(txid, newid)
transfer_data = bigchainAPI.queryTransaction(newid)
print(transfer_data)
test()
运行后
root@node1:~# python3 bigchainAPI.py
txid is 6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5
trials 1
trials 2
Tx valid in: 2 secs
>>> txid is 6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5
>>> data retrieved from bigchainDB
{'data': {'author': 'Wu Luo'}}
txid is c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8
trials 1
trials 2
Tx valid in: 2 secs
6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5 c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8
{'id': '6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5'}部署成功