基于ChainMaker TEE部署

TEE需要硬件CPU支持Intel SGX，由于公司没有相关机器，所以使用阿里云服务器（注意必须是ecs.c7t.large 型号，也就是安全增强型）选用Ubuntu* 20.04系统

安装golang用于编译Tee

wget -c https://dl.google.com/go/go1.16.2.linux-amd64.tar.gz -O - | sudo tar -xz -C /usr/local
vim .profile 
export PATH=$PATH:/usr/local/go/bin
source ~/.profile
go env -w GOPROXY=https://goproxy.cn,direct

安装SGX

需要安装 Intel(R) SGX driver、Intel(R) SGX SDK和Intel(R) SGX Platform Software (PSW) 共3个，官方指南安装步骤如下

apt-get update
sudo apt-get install build-essential ocaml ocamlbuild automake autoconf libtool wget python-is-python3 libssl-dev git cmake perl
sudo apt-get install libssl-dev libcurl4-openssl-dev protobuf-compiler libprotobuf-dev debhelper cmake reprepro unzip
git clone https://github.com/intel/linux-sgx.git
cd linux-sgx && make preparation
// 以上都是参考官方文档进行环境配置

这里安装分为2种源码安装和 bin 安装，我采用的是bin安装

下载bin https://www.jianshu.com/p/f7a57aee9e96
只需要下载sgx_linux_x64_sdk_2.14.100.2.bin和sgx_linux_x64_driver_2.11.0_2d2b795.bin
一个是SDK 一个是driver

安装 sdk 和 driver (注安装路径必须在/opt/intel/)

 sudo chmod u+x sgx_linux_x64_driver_2.11.0_2d2b795.bin
 ./sgx_linux_x64_driver_2.11.0_2d2b795.bin
  sudo chmod u+x sgx_linux_x64_sdk_2.14.100.2.bin
 ./sgx_linux_x64_sdk_2.14.100.2.bin

以下是我安装好后的路径

root@iZ2zefgt7bxps4bisz90chZ:~# ls /opt/intel/
sgx-aesm-service  sgxdriver  sgxsdk

bin安装只有sdk 和 driver 但是缺少 PSW ，所以必须通过源码来安装PSW，通过官方文档安装步骤如下
```
make psw
make clean
make deb_psw_pkg
sudo apt-get install libssl-dev libcurl4-openssl-dev libprotobuf-dev
apt-get install libsgx-launch libsgx-urts
apt-get install libsgx-epid libsgx-urts
apt-get install libsgx-quote-ex libsgx-urts
apt-get install libsgx-dcap-ql
sudo service aesmd restart
```
如果没有出现错误提示基本表示 Intel(R) SGX 安装完成
接下来通过编写一个简单的Hello world 来验证 SGX 是否安装成功
请参考https://www.jianshu.com/p/efa1a39a9762 验证SGX环境没有问题后，继续如下

编译chainmaker-tee

这里就按照官方文档依次执行就好了

#### 下载Chainmaker-tee
git clone --recursive [email protected]:chainmaker/chainmaker-tee.git
#### 根据Enclave.edl生成Enclave_t.c等文件
cd Enclave
/opt/intel/sgxsdk/bin/x64/sgx_edger8r --untrusted ./Enclave.edl --search-path . --search-path /opt/intel/sgxsdk/include --search-path ./openssl/include

/opt/intel/sgxsdk/bin/x64/sgx_edger8r --trusted ./Enclave.edl --search-path . --search-path /opt/intel/sgxsdk/include --search-path ./openssl/include

###### 复制Enclave_u.c和Enclave_u.h到gateway/bridge下
手动删除 Enclave_u.h 中的第8行 :  #include "sgx_edger8r.h"
cp Enclave_u.* ../gateway/bridge
cp user_types.* ../gateway/bridge

###### 生成cgo代码
cd ../gateway/bridge
go tool cgo bridge.go

###### 编译Enclave
cd ../../Enclave
cmake .
make clean
make

###### 对libenclave.so进行签名
/opt/intel/sgxsdk/bin/x64/sgx_sign sign -key ./Enclave_private.pem -enclave libenclave.so -out ./enclave.signed.so -config ./Enclave.config.xml

##### 编译隐私计算网关
cd ../gateway
go build

执行完成后在chainmaker-tee/gateway下能看到一个gateway的可执行文件，最终如下图

root@iZ2zefgt7bxps4bisz90chZ:~/chainmaker-tee/gateway# ls
bridge      cmclient  config.yml  go.mod          loggers  modules         out_report.dat    routes            secretkey.pem      tee_cert.pem    tee_target.dat
CAcert.pem  cmd       controller  go.sum          logs     out_csr.pem     publickeyExt.pem  sdk_config.yml    service            tee_csr.dat     tools
cert        config    gateway     in_teecert.pem  main.go  out_pubkey.pem  publickey.pem     secretkeyExt.pem  sgxkeyrequest.dat  tee_report.dat  verify

其中CAcert.pem是CA根证书 in_teecert.pem 是 tee自签证书 out_csr.pem是证书请求文件，CAcert.pem和 in_teecert.pem 则是由自建CA生成，具体生成方法如下

自建私有CA

ChainMaker TEE 需要有第三方CA来签发隐私网关证书，这里采用自建CA来颁发证书，使用openssl 来自建CA

修改配置文件

cd /usr/lib/ssl
cp openssl.cnf openssl.cnf.bak
vim openssl.cnf

具体修改内容如下

[ ca ]
# `man ca`
default_ca = CA_default

[ CA_default ]
# Directory and file locations.
dir               = /root/ca
certs             = $dir/certs
crl_dir           = $dir/crl
new_certs_dir     = $dir
database          = $dir/index.txt
serial            = $dir/serial
RANDFILE          = $dir/.rand

# The root key and root certificate.
#private_key       = $dir/private/rsa_private_key.pem
private_key       = $dir/private/cakey.pem
certificate       = $dir/cacert/CAcert.pem

# For certificate revocation lists.
crlnumber         = $dir/crlnumber
crl               = $dir/crl/ca.crl.pem
crl_extensions    = crl_ext
default_crl_days  = 30

# SHA-1 is deprecated, so use SHA-2 instead.
default_md        = sha256

name_opt          = ca_default
cert_opt          = ca_default
default_days      = 375
preserve          = yes
policy            = policy_strict
copy_extensions   = copy
x509_extensions   = v3_req
unique_subject = no

[ policy_strict ]
# The root CA should only sign intermediate certificates that match.
# See the POLICY FORMAT section of `man ca`.
countryName             = optional
stateOrProvinceName     = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = optional
emailAddress            = optional

[ policy_loose ]
# Allow the intermediate CA to sign a more diverse range of certificates.
# See the POLICY FORMAT section of the `ca` man page.
countryName             = optional
stateOrProvinceName     = optional
localityName            = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = optional
emailAddress            = optional

[ req ]
# Options for the `req` tool (`man req`).
default_bits        = 2048
distinguished_name  = req_distinguished_name
string_mask         = utf8only

# SHA-1 is deprecated, so use SHA-2 instead.
default_md          = sha256

# Extension to add when the -x509 option is used.
#x509_extensions     = v3_ca

[ req_distinguished_name ]
# See .
countryName                     = CN
stateOrProvinceName             = SZ
x509_extensions                 = v3_ca

# Optionally, specify some defaults.
countryName_default             = CN
stateOrProvinceName_default     = ShenZhen
localityName_default            = ShenZhen
0.organizationName_default      = Alice Ltd
organizationalUnitName_default  = 
emailAddress_default            =

[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
copy_extensions = none

[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ usr_cert ]
# Extensions for client certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, email
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, emailProtection

[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always

[ ocsp ]
# Extension for OCSP signing certificates (`man ocsp`).
basicConstraints = CA:FALSE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, digitalSignature
extendedKeyUsage = critical, OCSPSigning

[ v3_req ]

# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment

其中特别注意一个地方必须是 copy_extensions = copy
创建ca root证书

cd /root/ca
mkdir newcerts private cacert
touch index.txt  #生成证书索引数据库文件
echo 01 > serial #指定第一个颁发证书的序列号
#生成私钥
openssl genrsa -out private/cakey.pem 2048
#生成自签名证书
openssl req -new -x509 -key private/cakey.pem -days 3650 -out cacert/CAcert.pem
#签署TEE证书  out_csr.pem 为tee的申请证书
openssl ca -in out_csr.pem -out in_teecert.pem -days 100

1.这里需要注意in_teecert.pem 这个名字不能变
2.这里生成的in_teecert.pem是全证书，但是我们只需要-----BEGIN CERTIFICATE----- 至-----END CERTIFICATE----- 之间的证书，删除多余部分
正确的in_teecert.pem 格式如下

接着我们将链起来，接着我们需要将Enclave可信区证明report文件(out_report.dat) 上链

root@iZ2zefgt7bxps4bisz90chZ:~/chainmaker-tee/gateway# cmc tee upload_report --sdk-conf-path=./sdk_config.yml  --report=./out_report.dat
command execute successfully.

将第三方CA的签名根证书上链,也就是自建的CA的CAcert.pem 上链

root@iZ2zefgt7bxps4bisz90chZ:~/chainmaker-tee/gateway# cmc tee upload_ca_cert --sdk-conf-path=./sdk_config.yml  --ca_cert=./CAcert.pem
command execute successfully.

注意将in_teecert.pem 拷贝到chainmaker-tee/gateway
最终chainmaker-tee/gateway 目录如下

root@iZ2zefgt7bxps4bisz90chZ:~/chainmaker-tee/gateway# ls
bridge      cmclient  config.yml  go.mod          loggers  modules         out_report.dat    routes          sdk.log.2021072817  service            tee_csr.dat     tools
CAcert.pem  cmd       controller  go.sum          logs     out_csr.pem     publickeyExt.pem  sdk_config.yml  secretkeyExt.pem    sgxkeyrequest.dat  tee_report.dat  verify
cert        config    gateway     in_teecert.pem  main.go  out_pubkey.pem  publickey.pem     sdk.log         secretkey.pem       tee_cert.pem       tee_target.dat

启动隐私网关

root@iZ2zefgt7bxps4bisz90chZ:~/chainmaker-tee/gateway# ./gateway start
2021/07/28 17:33:46 starting sgx server
###### report = BQUICf//AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BwAAAAAAAADnAAAAAAAAAA154rx++VrK3cPheTLzr+nmLoydXDtfvu0V7beSKT72
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACD1xnnferKFHD2uvYqTXdD
A8iZ22kCD5xw7h38CMfOngAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
jJkOXRlrl0tVwibN8eDgNwAAAAAAAAAAAAAAAAAAAAAUBysfiiP97+mmb+mPzbSr

###### report from chain = BQUICf//AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BwAAAAAAAADnAAAAAAAAAA154rx++VrK3cPheTLzr+nmLoydXDtfvu0V7beSKT72
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACD1xnnferKFHD2uvYqTXdD
A8iZ22kCD5xw7h38CMfOngAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
jJkOXRlrl0tVwibN8eDgNwAAAAAAAAAAAAAAAAAAAAAUBysfiiP97+mmb+mPzbSr

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /private/deploy           --> gateway/controller.Deploy (2 handlers)
[GIN-debug] POST   /private/compute          --> gateway/controller.Compute (2 handlers)
[GIN-debug] POST   /private/remote_attestation --> gateway/controller.RemoteAttestation (2 handlers)
[GIN-debug] GET    /healthy                  --> gateway/controller.Healthy (2 handlers)
2021/07/28 17:33:49 Server Run http://0.0.0.0:8081/ 
2021/07/28 17:33:49 Enter Control + C Shutdown Server