edgexfoundry 安全模块实现详解 vault kong (shell版)

一、安全功能流程概述
    安全功能主要涉及到如下几个模块：vault, vault-worker , security-api-gateway, kong 

    下面是各模块功能介绍：
 

•     vault：生成root CA 证书与私钥、edgex-vault证书与私钥（支持https用）
•     vault-worker：(若不存在)生成kong证书与私钥，并保存到vault中去；向consul发起心跳检测，看vault是否存活
•     security-api-gateway：根据token，从vault服务器中获取证书，并上传到kong服务器；管理kong用户、生成JWT字串
•     kong：实现edgexfoundry各微服务的安全访问(获取了JWT，通JWT字串访问后端微服务)

 

   1、vault启动时，生成root CA 证书与私钥，并生成edgex-vault证书与私钥

 

   2、worker启动时，每定时（3分钟）执行如下动作

 

      1.1、探测试vault是否活着，并注册到consul

 

      1.2、上面vault活着的话，则执行vault-kong.sh脚本，它主要动作如下：

 

           1. 2.1 看是vault服务器上是否存在edgex-kong证书（如果本地edgex-kong证书没有，则先生成edgex-kong证书），没有的话就上传到vault

   

   3、security-api-gateway ：初始化kong（从vault获取证书密钥，并上传到kong）

二、功能代码详细解读


2.1 镜像edgexfoundry/docker-edgex-vault制作过程
   

2.1.1  Dockerfile.vault文件

#vault官方基础镜像

FROM vault:0.10.2

LABEL license='SPDX-License-Identifier: Apache-2.0' \
      copyright='Copyright (c) 2018: ForgeRock'

USER root
WORKDIR /vault/config
#使用tls 方式，将vault注册到consul

COPY local-tls.json ./local.json
RUN chmod 777 /vault/config/local.json

# Vault PKI/TLS setup/config and X.509 materials
WORKDIR /vault
COPY pki-setup.sh .
COPY pki-setup-config-vault.env .

# install pre-requisites for pki setup: bash/openssl, pki/tls setup and housekeeping
RUN apk --no-cache update && \
    apk --no-cache add bash openssl && \   #安装bash openssl工具，用于生成证书
    chown -R vault:vault /vault && \
    chmod 777 /vault/config/local.json && \
    chmod 744 pki-setup* && \
    ./pki-setup.sh pki-setup-config-vault.env && \    #生成根证书密钥，及vault证书密钥，后面会详述
    chown -R vault:vault /vault/pki && \
    chmod 750 /vault/pki && \
    chmod 640 /vault/pki/*/* && \ 
    apk del --purge bash openssl && \  #安全起见，删除bash openssl工具、删除脚本，也可以减小镜像的空间大小
    rm -f /vault/pki-setup.sh /vault/pki-setup-config-vault.env && \
    rm -f /var/cache/apk/*

# Set the default path for Docker exec sessions
WORKDIR /

我们结合docker-compose.yml文件，看vault服务一段，如下

  vault:
    image: edgexfoundry/docker-edgex-vault:security
    container_name: edgex-vault
    hostname: edgex-vault
    networks:
      - edgex-network
    ports:
      - "8200:8200"
    cap_add:
      - "IPC_LOCK"
    command: "server" #vault容器启动后，执行server命令，拉起vault服务
    environment:
      - 'VAULT_ADDR=https://edgex-vault:8200'  #vault地外服务地址
      - 'VAULT_CONFIG_DIR=/vault/config' # vault启动配置文件，镜像制作是local.json已拷贝到此目录
      - 'VAULT_UI=true'  # 开启UI 。在浏览器访问https://localhost:8200/ui ,可打开管理界面
    volumes:
      - vault-config:/vault/config
      - vault-pki:/vault/pki
      - vault-file:/vault/file
      - vault-logs:/vault/logs
    depends_on:
      - volume
      - consul

 


2.1.1  local-tls.json文件


#tcp监听配置

listener "tcp" { 
  address = "edgex-vault:8200" #访问地址
  tls_disable = "0" #开启tls，即https访问
  cluster_address = "edgex-vault:8201" #集群地址
  tls_min_version = "tls12" 
  tls_client_ca_file ="/vault/pki/EdgeXFoundryTrustCA/EdgeXFoundryTrustCA.pem"   #客户端访问的证书
  tls_cert_file ="/vault/pki/EdgeXFoundryTrustCA/edgex-vault.pem"        #支持https用的证书   
  tls_key_file = "/vault/pki/EdgeXFoundryTrustCA/edgex-vault.priv.key"   #支持https用的密钥
}


#注册consul的配置
backend "consul" {
  path = "vault/"   #consul的services显示的名字
  address = "edgex-core-consul:8500"  #consul服务地址
  scheme = "http"  #注册使用的协议
  redirect_addr = "https://edgex-vault:8200"  #真实的地址，对应上段的address ，通过consul访问vault时，转发到此地址
  cluster_addr = "https://edgex-vault:8201"   # 对应上段的cluster_address ，意义同上行
}

default_lease_ttl = "168h"  
max_lease_ttl = "720h"

2.1.2  pki-setup.sh 与 pki-setup-config-vault.env文件 

在Dockerfile.vault 文件中，有一行  ./pki-setup.sh pki-setup-config-vault.env && \ ，下面了解一下执行过程。


2.1.2.1 pki-setup.sh文件


#!/bin/bash

#  pki-setup.sh 用法，即help

function usage() {
        echo "Usage  : pki-setup.sh "
        echo "Example: pki-setup.sh pki-setup-config.env"
        echo " "
}

function initEnv() {
         # --------------------------------------------
         DEBUG=$1  # Debug switch: "debug-off" or "debug-on"
         #openssl是否存在
         OPENSSL_BIN=$(which openssl)
         if [[ $? == 1 ]]; then
            echo "ERROR: openssl binary not found or not in the path"
            exit
         fi
         echo ">> openssl binary found: ${OPENSSL_BIN}"
         echo ">>" $(openssl version)
         echo ""
         # --------------------------------------------
         #将pki-setup-config-vault.env 或  pki-setup-config-kong.env里定义的内容装入到shell变量
         source $1  #
         # --------------------------------------------
}

function initCA() {
         # ---------------------------------------------
         # CA (self signing) name, keystore alias and file (.pem .priv.key)
         CA_NAME=${ORG}$1  #值为：EdgeXFoundryTrustCA
         # CA root certificate subject
         # CA_C
         # CA_ST
         # CA_L
         # CA_O
         CA_CN=${ORG}" "$1 #值为：EdgeXFoundry TrustCA
         CA_HOME=${SETUP_PKI}/${CA_NAME} #值为：/vault/pki/EdgeXFoundryTrustCA
         CERT_EXTENSIONS="ssl_server.ext"    #扩展文件，在签发证书请求（signCertReq）时需要
         
         if [[ ${CREATE_CA} == "true" ]]; then #setup vault需要，kong时不需要
                # Create configuration directory (CA_HOME)
                echo ">> Creating a new Root CA Certificate"
                rm -rf ${CA_HOME}
                mkdir -p ${CA_HOME}  #创建目录：/vault/pki/EdgeXFoundryTrustCA
                chmod -R 755 ${SETUP_PKI}
                # Generate an initial serial number for signed/generated certificates
                echo `date +%s` > ${CA_HOME}/${CA_NAME}.srl    #以当前时间，生成初始序号，放入EdgeXFoundryTrustCA.srl文件，在签发证书求（signCertReq）时需要
                # Create a certificate extensions file:
                #  The generated server certificate will be for TLS server usage only!
                cat > ${CA_HOME}/${CERT_EXTENSIONS} <  #向扩展ssl_server.ext文件里写内容
nsCertType=server   #证书类型为server
EOF
        else
                echo ">> Reusing existing Root CA Certificate"
                # Certificate serial number is incremented during CA signing process
        fi
         # ---------------------------------------------
}

#定义FQDN ，当DOMAIN为local时，FQDN 值为 HOST (即：edgex-vault)，否则为 edgex-vault.XXX  (XXX 代表DOMAIN), 

#本例FQDN 为edgex-vault 或 edgex-kong ，此值在生成证书求证时要用到，作为CN的值
function initVaultServer() {
         # --------------------------------------------
         # Vault FQDN, files (.pem .req .priv.key)
         if [[ ${DOMAIN} == "local" ]]; then
                FQDN=$1 
         else
                FQDN=$1.${DOMAIN}
         fi
         # Vault TLS certificate subject
         # TLS_C
         # TLS_ST
         # TLS_L
         # TLS_O
         TLS_CN=${FQDN}
         # ---------------------------------------------
}

#创建CA ,生成EdgeXFoundryTrustCA.pem 与 EdgeXFoundryTrustCA.priv.key 文件
function genRootCA() {
        # Generate Root CA private key (RSA/4096) and trusted Root CA certificate (1825 days = 5 years)
        ${OPENSSL_BIN} req -x509 -sha256 -nodes -days 1825 -newkey rsa:4096 \
                   -subj "/C=${CA_C}/ST=${CA_ST}/L=${CA_L}/O=${CA_O}/CN=${CA_CN}/emailAddress=${CA_NAME}@${DOMAIN}"  \
                   -keyout ${CA_HOME}/${CA_NAME}.priv.key -out ${CA_HOME}/${CA_NAME}.pem

        # Generate Root CA private key (ECDSA/secp384r1) and trusted Root CA certificate
        #${OPENSSL_BIN} req -x509 -sha256 -nodes -days 1825 -newkey ec:secp384r1 \
        #           -subj "/C=${CA_C}/ST=${CA_ST}/L=${CA_L}/O=${CA_O}/CN=${CA_CN}/emailAddress=${CA_NAME}@${DOMAIN}"  \
        #           -keyout ${CA_HOME}/${CA_NAME}.priv.key -out ${CA_HOME}/${CA_NAME}.pem

        if [[ ${DEBUG} == "debug-on" ]]; then
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                echo " CA Signing Certificate"
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                ${OPENSSL_BIN} x509 -in ${CA_HOME}/${CA_NAME}.pem -noout -text
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
        fi
}

#生成edgex-vault.req 或 edgex-kong.req证书请求文件
function genVaultCertReq() {
        # Generate TLS server private key (RSA/4096)
        ${OPENSSL_BIN} genrsa -out ${CA_HOME}/${FQDN}.priv.key 4096

        # Generate TLS server private key (ECDSA/secp384r1)
        #${OPENSSL_BIN} ecparam -genkey -name secp384r1 -out ${CA_HOME}/${FQDN}.priv.key

        # Generate server certificate signing request for TLS server certificate (CN must equal the FQDN)
        ${OPENSSL_BIN} req -sha256 -new -key ${CA_HOME}/${FQDN}.priv.key \
                   -out ${CA_HOME}/${FQDN}.req \
                   -subj "/C=${TLS_C}/ST=${TLS_ST}/L=${TLS_L}/O=${TLS_O}/CN=${TLS_CN}/emailAddress=admin@${DOMAIN}" #TLS_CN为edgex-vault

        if [[ ${DEBUG} == "debug-on" ]]; then
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                echo " TLS Server Certificate Request for: ${FQDN}"
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                ${OPENSSL_BIN} req -in ${CA_HOME}/${FQDN}.req -noout -text
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
        fi
}

#签发证书，生成edgex-vault.pem edgex-vault.priv.key 或 edgex-kong.pem edgex-kong.priv.key
#输入证书请求文件（edgex-vault.req 或 edgex-kong.req）, CA 的扩展文件 与 CA的序列文件srl
function signVaultCertReq() {
        # CA signs TLS server certificate request with its trusted root certificate,
        # and creates the final TLS server certificate (1825 days = 5 years)
        ${OPENSSL_BIN} x509 -sha256 -req -in ${CA_HOME}/${FQDN}.req \
                    -CA ${CA_HOME}/${CA_NAME}.pem -CAkey ${CA_HOME}/${CA_NAME}.priv.key \
                    -CAserial ${CA_HOME}/${CA_NAME}.srl \
                    -extfile ${CA_HOME}/${CERT_EXTENSIONS} \
                    -days 1825 -outform PEM -out ${CA_HOME}/${FQDN}.pem

        if [[ ${DEBUG} == "debug-on" ]]; then
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                echo " TLS Server Certificate for: ${FQDN}"
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                ${OPENSSL_BIN} x509 -in ${CA_HOME}/${FQDN}.pem -noout -text
                echo "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
        fi
}


#将签发后证书与密钥，存储为PKCS12格式，此格式的好处是需要密码访问
#即生成 edgex-kong.p12 或 edgex-vault.p12 文件
function storeToPKCS12() {
        # Create PKCS12 container for TLS server certificate and SK 
        # SK: Secret Key aka Private key
        # PK: Public Key
        #
        ${OPENSSL_BIN} pkcs12 -export -in ${CA_HOME}/${FQDN}.pem \
                      -inkey ${CA_HOME}/${FQDN}.priv.key -out ${CA_HOME}/${FQDN}.p12 \
                      -name ${FQDN} -password pass{PKCS12_PASSWORD}
}

function houseKeeping() {
        chmod 644 ${SETUP_PKI}/${CA_NAME}/*
}
# ----------------- Common Functions ------------------------

#===================================== MAIN ====================================

# pki-setup.sh arguments check
#  检测pki-setup.sh 后是否有参数 ，没有的话，若调用usage函数显法用法
if [[ $# != 1 ]]; then
        echo "SYNTAX ERROR"
        usage
        exit
fi

#  检测pki-setup.sh 后是的参数文件是否存在，即pki-setup-config-vault.env 或  pki-setup-config-kong.env 
if [[ ! -f $1 ]]; then
        echo "FILE $1 NOT FOUND"
        exit
fi

# 环境初始化：判断openssl是否安装、将pki-setup-config-vault.env 或  pki-setup-config-kong.env里定义的内容装入到shell变量
echo ">> Create binary paths and folder paths"
initEnv $1

echo ">> Prepare Root CA Certificate"
# 初始化CA： 1、创建/vault/pki/EdgeXFoundryTrustCA目录 2、准备好 EdgeXFoundryTrustCA.srl 与 ssl_server.ext，它们用于签发（子）证书
initCA TrustCA

echo ">> Prepare Vault Server TLS certificate"
#定义FQDN ，当DOMAIN为local时，FQDN 值为 HOST (即：edgex-vault 或 edgex-kong)，否则为 edgex-vault.XXX  (XXX 代表DOMAIN), 

#本例FQDN 的值为edgex-vault 或 edgex-kong ，此值在生成证书求证时要用到，作为CN的值
initVaultServer ${HOST}  #HOST值是：edgex-vault 或 edgex-kong

if [[ ${CREATE_CA} == "true" ]]; then  #edgex-vault 需要创建CA 
        echo ">> Generate Root CA private key and trusted root certificate"
        genRootCA debug-off  #创建CA ,生成EdgeXFoundryTrustCA.pem 与 EdgeXFoundryTrustCA.priv.key 文件
fi

echo ">> Generate TLS server private key"
echo ">> Generate server certificate signing request for TLS server certificate (CN must equal the FQDN)"

#生成edgex-vault.req  或 edgex-kong.req证书请求文件

genVaultCertReq debug-off

echo ">> Root CA signs TLS server certificate request with its trusted root certificate,"
echo ">> and creates the final TLS server certificate"

#签发证书，生成edgex-vault.pem edgex-vault.priv.key 或 edgex-kong.pem edgex-kong.priv.key
#输入证书请求文件（edgex-vault.req 或 edgex-kong.req）, CA 的扩展文件 与 CA的序列文件srl
signVaultCertReq debug-off

echo ">> Create PKCS12 containers for TLS server certificate and SK"
#将签发后证书与密钥，存储为PKCS12格式，此格式的好处是需要密码访问
#即生成 edgex-kong.p12 或 edgex-vault.p12 文件
storeToPKCS12


#安全起见，限制目录权限/vault/pki/EdgeXFoundryTrustCA
houseKeeping
echo ">> PKI setup script completed."

exit

#EOF


===========================================



vault镜像从上面文件，可以归纳实现了如下功能：

• 自建了CA，并用此CA签发了vault证书，另外一个kong证书也类似生成
• local-tls.json定义了vault采用tls 方式访问,注册到consul

 

2.2 镜像edgexfoundry/docker-edgex-vault-worker制作过程

2.2.1  Dockerfile.vault-worker文件
#使用alpine基础镜像，这是个很小的linux操作系统，只有几M大
FROM alpine:latest

LABEL license='SPDX-License-Identifier: Apache-2.0' \
      copyright='Copyright (c) 2018: ForgeRock'

USER root

# Vault init and unseal scripts
WORKDIR /vault

# Copy main launcher [CMD]
COPY vault-worker.sh .
# Copy init-unseal process
COPY vault-init-unseal.sh .
# Copy PKI/TLS script/config for Kong
COPY pki-setup.sh .
COPY pki-setup-config-kong.env .
# Copy PKI/TLS setup and Vault import for Kong
COPY vault-kong.sh .

# Create a vault user and group so the IDs get set the same way
# Set the files ownership and rights
# Install pre-requisites for pki setup: bash/curl/jq/dig/openssl
RUN addgroup vault && \
    adduser -S -G vault vault && \
    chmod 700 vault-*.sh pki-setup* && \
    chown vault:vault vault-*.sh pki-setup* && \
    apk --no-cache update && \
    apk --no-cache add bash curl jq bind-tools openssl && \
    rm -f /var/cache/apk/*

# Set the operation command
ENTRYPOINT ["/bin/bash", "-c"]
#容器启动后执行vault-worker.sh
CMD ["/vault/vault-worker.sh"] 


2.2.2  vault-worker.sh文件

#!/bin/bash
while true
do
   # Init/Unseal processes
   /vault/vault-init-unseal.sh

   # If Vault init/unseal was OK... eventually prepare materials for Kong
   if [[ $? == 0 ]]; then 
       /vault/vault-kong.sh # 如果/vault/vault-init-unseal.sh执行成功则执行此步
   fi

   sleep ${WATCHDOG_DELAY}  #此变量来自docker-compose.yaml文件中的environment定义，定为3分种循环执行一次
done

exit

#EOF




我们结合docker-compose.yml文件，看 vault-worker服务一段，如下
vault-worker:
    image: edgexfoundry/docker-edgex-vault-worker:security
    container_name: edgex-vault-worker
    hostname: edgex-vault-worker
    networks:
      - edgex-network
    environment:
     - 'WATCHDOG_DELAY=3m'    #这个在vault-worker.sh有引用
    volumes:
      - vault-pki:/vault/pki
      - vault-file:/vault/file
    depends_on:
      - volume
      - consul
      - vault  


2.2.3  vault-init-unseal.sh文件

#功能：实现vault初始化、启封、检查vault是否注册到consul


#!/bin/bash
# ----------------- Common Functions ------------------------
#
function houseKeeping() {
    rm -f ${_TMP}
    rm -f ${_PAYLOAD_INIT}
    rm -f ${_PAYLOAD_UNSEAL}
}

#初始化vault
function vaulInitialization() {
    echo ">> (1) Vault Initialization Process"

    # Create the Vault init request payload with only 1 key and obviously threshold 1
#创建payload-init.json文件，写入如下内容

    cat > ${_PAYLOAD_INIT} <
    {
    "secret_shares": 1,
    "secret_threshold": 1
    }
EOF
    # ---------------------------------------------------------------------------
    # Vault Initialization API
    # ---------------------------------------------------------------------------
    curl -sw 'HTTP-STATUS: %{http_code}\n' ${_TLS} ${_REDIRECT} \    
        --request PUT \
        --data @${_PAYLOAD_INIT} \   #
        ${_HTTP_SCHEME}://${_VAULT_SVC}{_VAULT_PORT}/v1/sys/init > ${_TMP}
   #上面命令真实值为：curl sw 'HTTP-STATUS: %{http_code}\n'  --cacert /vault/pki/EdgeXFoundryTrustCA/EdgeXFoundryTrustCA.pem  --location \
    #                                        --request PUT \
    #                                        --data /vault/file/payload-init.json https://edgex-vault:8200/v1/sys/init > /vault/file/_tmp.vault


    #说明： https://edgex-vault:8200 在docker-compose.yaml中的vault服务中environment 定义：VAULT_ADDR
    #         EdgeXFoundryTrustCA.pem 在local-tls.json中tls_client_ca_file定义的要一致
    }
    # ---------------------------------------------------------------------------

    # Check http status code returned by init request
    result=$(tail -1 ${_TMP} | grep "HTTP-STATUS:" | cut -d' ' -f2)

    case ${result} in
        # If Vault initialization OK
        #
        # 执行上面的curl后，若初始化成功，会返回类似如下的字串
        # {"keys":["8e70bcf6ba046b59857cba1ec6495c58b53f6c60e37f871e53b4b391ff43ec59"],"keys_base64":["jnC89roEa1mFfLoexklcWLU/bGDjf4ceU7Szkf9D7Fk="],"root_token":"6e2e099f-e5a4-028b-6b84-f11d1fb1ad9d"}
        # http响应状态码为200时，说明成功，将返回内容中的keys的值放入_INIT_KEY ，root_token的值放入 _ROOT_TOKEN
        "200")
            # let's grab the init key
            _INIT_KEY=$(head -1 ${_TMP} | jq -r '.keys | .[0]')
            # let's grab the root token
            _ROOT_TOKEN=$(head -1 ${_TMP} | jq -r '.root_token')
            # save the key and the root token JSON (strip HTTP-STATUS)
#resp-init.json存放类似 {"keys":["8e70bcf6ba046b59857cba1ec6495c58b53f6c60e37f871e53b4b391ff43ec59"],"keys_base64":["jnC89roEa1mFfLoexklcWLU/bGDjf4ceU7Szkf9D7Fk="],"root_token":"6e2e099f-e5a4-028b-6b84-f11d1fb1ad9d"}
            head -1 ${_TMP} | jq '.' > ${_RESP_INIT} 
            chown vault:vault ${_RESP_INIT}
            echo ">> Vault successfully initialized"
        ;;
        # If Vault already initialized
        #
        # Example:
        # {"errors":["Vault is already initialized"]}
         # http响应状态码为400时，说明Vault 已经初始化过，直接从resp-init.json取出_INIT_KEY 与 _ROOT_TOKEN
        "400")
            # let's grab the error message in case...
            result=$(head -1 ${_TMP} | jq -r '.errors | .[0]')
            echo ">> ${result}"
            # let's go unseal... but before we need:
            # 1) Check previous init response JSON still exists
            if [[ -f ${_RESP_INIT} ]]; then
                # 2) Grab the init key from previous init
                _INIT_KEY=$(cat ${_RESP_INIT} | jq -r '.keys | .[0]')
                # 3) Grab the root token from previous init
                _ROOT_TOKEN=$(cat ${_RESP_INIT} | jq -r '.root_token')
            else
                echo ">> Vault initialization error!"
                echo ">> Previous init response file not found: ${_RESP_INIT}"
                _EXIT="1"
            fi
        ;;
        #
        # Unattended error...
        #
        *)
            echo ">> Vault initialization unattended error"
            _EXIT="1"
        ;;
    esac

    # Handle the error exit use case for Vault init process
    if [[ ${_EXIT} == "1" ]]; then
        echo "==> Vault init request response:"
        cat ${_TMP}
        echo ">>"
    fi

    return ${_EXIT}
}

# vault初始化后，默认是密封的，需要正确启封才可用
function vaultUnsealing() {
    echo ">> (2) Vault Unseal Process"

    # https://www.vaultproject.io/api/system/seal-status.html

    # Create the Vault unseal request payload with the unseal key
  #创建payload-unseal.json文件，写入{ "key": "${_INIT_KEY}"}
    cat > ${_PAYLOAD_UNSEAL} <
    {
    "key": "${_INIT_KEY}" #初始化vault时获理的值_INIT_KEY
    }
EOF
    # ---------------------------------------------------------------------------
    # Vault Unsealing API
    # ---------------------------------------------------------------------------
    curl -sw 'HTTP-STATUS: %{http_code}\n' ${_TLS} ${_REDIRECT} \
        --request PUT \
        --data @${_PAYLOAD_UNSEAL} \
        ${_HTTP_SCHEME}://${_VAULT_SVC}{_VAULT_PORT}/v1/sys/unseal > ${_TMP}

   #上面命令真实值为：curl sw 'HTTP-STATUS: %{http_code}\n'  --cacert /vault/pki/EdgeXFoundryTrustCA/EdgeXFoundryTrustCA.pem  --location \
    #                                        --request PUT \
    #                                        --data /vault/file/payload-unseal.json https://edgex-vault:8200/v1/sys/init > /vault/file/_tmp.vault



    # ---------------------------------------------------------------------------

    # Check http status code returned by unseal request
    result=$(tail -1 ${_TMP} | grep "HTTP-STATUS:" | cut -d' ' -f2)

    # If Vault unsealing OK
    #
    # Example:
    # {"type":"shamir","sealed":false,"t":1,"n":1,"progress":0,"nonce":"","version":"0.10.2","cluster_name":"vault-cluster-1df0f671","cluster_id":"10f1a1eb-ad7a-511f-06af-aab9c370e412"}
    # HTTP-STATUS: 200
    #
    # Remark: unsealing Vault when already unsealed generates same output response with code 200
     # http响应状态码为200时，说明成功
    if [[ ${result} == "200" ]]; then
        # let's grab the sealed state boolean
        result=$(head -1 ${_TMP} | jq -r '.sealed')
        if [[ ${result} == "false" ]]; then #"sealed":false 说明启封状态，可以用了
            # save the unseal JSON response (strip HTTP-STATUS)
#保存到resp-unseal.json，内容类似 {"type":"shamir","sealed":false,"t":1,"n":1,"progress":0,"nonce":"","version":"0.10.2","cluster_name":"vault-cluster-1df0f671","cluster_id":"10f1a1eb-ad7a-511f-06af-aab9c370e412"}
            head -1 ${_TMP} | jq '.' > ${_RESP_UNSEAL} #保存到resp-unseal.json
            chown vault:vault ${_RESP_UNSEAL}
            echo ">> Vault successfully unsealed"
        else
            echo ">> Vault unseal ok but incoherent sealed status!"
            _EXIT="1"
        fi
    else
        echo ">> Vault unseal error!"
        _EXIT="1"
    fi

    # Handle the error exit use case for Vault unseal process
    if [[ ${_EXIT} == "1" ]]; then
        echo "==> Vault unseal request response:"
        cat ${_TMP}
        echo ">>"
    fi

    return ${_EXIT}
}

# 检测vault是否成功注册到consul 
function vaultRegistered() {
    sleep 3 # Allow Consul DNS table refresh
    echo ">> Check Vault is registered as a service in Consul"
    echo ">> DNS requests to Consul service on port 8600/tcp"
    echo -n "vault.service.consul: "
    dig +short +tcp -p8600 @edgex-core-consul vault.service.consul #正常时返回“edgex-vault.”
    echo -n "active.vault.service.consul: "
    dig +short +tcp -p8600 @edgex-core-consul  active.vault.service.consul #正常时返回“edgex-vault.”
    echo ""

    return ${_EXIT}  #此段始终返回0，这个可能有问题，没有对上面的输出判断来改变_EXIT的值
}
# ----------------- Common Functions ------------------------


#===================================== MAIN INIT ===============================

# Variables and parameters
_VAULT_DIR="/vault"
_VAULT_CONFIG_DIR="${_VAULT_DIR}/config"
_VAULT_PKI_DIR="${_VAULT_DIR}/pki"
_VAULT_FILE_DIR="${_VAULT_DIR}/file"

_PAYLOAD_INIT="${_VAULT_FILE_DIR}/payload-init.json"
_PAYLOAD_UNSEAL="${_VAULT_FILE_DIR}/payload-unseal.json"
_RESP_INIT="${_VAULT_FILE_DIR}/resp-init.json"
_RESP_UNSEAL="${_VAULT_FILE_DIR}/resp-unseal.json"
_VAULT_CONFIG="${_VAULT_CONFIG_DIR}/local.json"
_TMP="${_VAULT_FILE_DIR}/_tmp.vault"
_EXIT="0"

_CA="EdgeXFoundryTrustCA"
_CA_DIR="${_VAULT_PKI_DIR}/${_CA}"
_CA_PEM="${_CA_DIR}/${_CA}.pem"
_TLS=" --cacert ${_CA_PEM}"
_REDIRECT=" --location" # If HTTP temporary redirect (HTTP STATUS 307) follow it
_HTTP_SCHEME="https"
_VAULT_SVC="edgex-vault"
_EDGEX_DOMAIN=""
_VAULT_PORT="8200"

#===================================== MAIN PROC ===============================

# 0) Cleanup up previous temp files and payloads
houseKeeping

# 1) Init Vault
#初始化Vault

vaulInitialization

# Handle the error exit use case for Vault init process
if [[ $? == "0" ]]; then
    # 2) Unseal Vault
    #Vault 启封

    vaultUnsealing 
    # Handle the error exit use case for Vault unseal process
    if [[ $? == "0" ]]; then
        # 4) Check Vault was successfully registered as a Consul service
        vaultRegistered 
    fi
fi

# 5) Cleanup up temp files and payloads
houseKeeping 

exit ${_EXIT}
#EOF



进入vault容器命令

1. docker exec -it bb28361c0bac /bin/sh

复制代码

 

token存储在：/vault/file/resp-init.json 
 
 

2.2.4  vault-kong.sh文件

#功能：将 edgex-kong.pem、edgex-kong.priv.key 存储到vault中

#!/bin/bash
function houseKeeping() {
    rm -f ${_TMP}
    rm -f ${_PAYLOAD_KONG}
}
# ----------------- Common Functions ------------------------

#===================================== MAIN ====================================

# Variables and parameters
_VAULT_DIR="/vault"
_VAULT_CONFIG_DIR="${_VAULT_DIR}/config"
_VAULT_PKI_DIR="${_VAULT_DIR}/pki"
_VAULT_FILE_DIR="${_VAULT_DIR}/file"

_PAYLOAD_INIT="${_VAULT_FILE_DIR}/payload-init.json"
_PAYLOAD_UNSEAL="${_VAULT_FILE_DIR}/payload-unseal.json"
_PAYLOAD_KONG="${_VAULT_FILE_DIR}/payload-kong.json"
_RESP_INIT="${_VAULT_FILE_DIR}/resp-init.json"
_RESP_UNSEAL="${_VAULT_FILE_DIR}/resp-unseal.json"
_VAULT_CONFIG="${_VAULT_CONFIG_DIR}/local.json"
_TMP="${_VAULT_FILE_DIR}/_tmp.vault"
_EXIT="0"

_CA="EdgeXFoundryTrustCA"
_CA_DIR="${_VAULT_PKI_DIR}/${_CA}"
_CA_PEM="${_CA_DIR}/${_CA}.pem"
_TLS=" --cacert ${_CA_PEM}"

_KONG_SVC="edgex-kong"
_KONG_PEM="${_CA_DIR}/${_KONG_SVC}.pem"
_KONG_SK="${_CA_DIR}/${_KONG_SVC}.priv.key"
_REDIRECT=" --location" # If HTTP temporary redirect (HTTP STATUS 307) follow it
_HTTP_SCHEME="https"
_VAULT_SVC="edgex-vault"
_EDGEX_DOMAIN=""
_VAULT_PORT="8200"
_VAULT_API_PATH_KONG="/v1/secret/edgex/pki/tls/${_KONG_SVC}"

houseKeeping # temp files and payloads

# Generate Kong PKI/TLS materials if they haven't been already...
#如果edgex-kong.pem 或 edgex-kong.priv.key 文件不存在，则由CA创建签发

if [[ (! -f ${_KONG_PEM}) || (! -f ${_KONG_SK}) ]]; then
    echo ">> (3) Create PKI materials for Kong TLS server certificate"
    /vault/pki-setup.sh /vault/pki-setup-config-kong.env
    chown vault:vault ${_CA_DIR}/${_KONG_SVC}.*
else
    echo ">> (3) PKI materials for Kong TLS server certificate already created"
    #查看证书信息

    openssl x509 -noout -subject -in ${_KONG_PEM}
    openssl x509 -noout -issuer -in ${_KONG_PEM}
fi

echo ""    
echo ">> (4) Fetch the Vault Root Token"
#从resp-init.json文件中获取Vault Root Token
_ROOT_TOKEN=$(cat ${_RESP_INIT} | jq -r '.root_token')

echo ">> (5) Test if the Kong Key/Value already exists"
#检测Kong在vault中是否存在
curl -sw 'HTTP-STATUS: %{http_code}\n' ${_TLS} ${_REDIRECT} \
    --header "X-Vault-Token: ${_ROOT_TOKEN}" \
    --request GET \
    ${_HTTP_SCHEME}://${_VAULT_SVC}{_VAULT_PORT}${_VAULT_API_PATH_KONG} > ${_TMP}

   #上面命令真实值为：curl sw 'HTTP-STATUS: %{http_code}\n'  --cacert /vault/pki/EdgeXFoundryTrustCA/EdgeXFoundryTrustCA.pem  --location \
    #                                        --header "X-Vault-Token: 6e2e099f-e5a4-028b-6b84-f11d1fb1ad9d" \
    #                                        --request GET \
    #                                        https://edgex-vault:8200/v1/secret/edgex/pki/tls/edgex-kong > /vault/file/_tmp.vault

# Check http status code returned by get request
result=$(tail -1 ${_TMP} | grep "HTTP-STATUS:" | cut -d' ' -f2)

case ${result} in
    # HTTP-STATUS: 200 -> key found
    "200")
        echo "==> Key/Value already in Vault, done!"
    ;;
    # HTTP-STATUS: 404 -> Key not found
    #如果Kong不存在vault中，则post edgex-kong.pem、edgex-kong.priv.key到vault中去（存储起来）
    "404")
        echo ">> (6) Create the Kong JSON with TLS certificate and private key (base64 encoded)"
        jq -n --arg cert "$(cat ${_KONG_PEM}|base64)" \  #将edgex-kong.pem base64编码后赋给变量cert 
            --arg sk "$(cat ${_KONG_SK}|base64)" \             #将edgex-kong.priv.key base64编码后赋给变量sk
            '{certcert,keysk}' > ${_PAYLOAD_KONG}  #生成payload-kong.json

        echo ">> (7) Load the Kong JSON PKI materials in Vault"
        echo "$(cat ${_PAYLOAD_KONG})"

        curl -sw 'HTTP-STATUS: %{http_code}\n' ${_TLS} ${_REDIRECT} \
            --header "X-Vault-Token: ${_ROOT_TOKEN}" \
            --header "Content-Type: application/json" \
            --request POST \
            --data @${_PAYLOAD_KONG} \
            ${_HTTP_SCHEME}://${_VAULT_SVC}{_VAULT_PORT}${_VAULT_API_PATH_KONG} > ${_TMP}
  #上面命令真实值为：curl sw 'HTTP-STATUS: %{http_code}\n'  --cacert /vault/pki/EdgeXFoundryTrustCA/EdgeXFoundryTrustCA.pem  --location \
    #                                        --header "X-Vault-Token: 6e2e099f-e5a4-028b-6b84-f11d1fb1ad9d" \
   #                                        --header "Content-Type: application/json" \    #                                        --request POST \
    #                                        --data /vault/file/payload-kong.json \
    #                                        https://edgex-vault:8200/v1/secret/edgex/pki/tls/edgex-kong > /vault/file/_tmp.vault


        # Check http status code returned by post request
        result=$(tail -1 ${_TMP} | grep "HTTP-STATUS:" | cut -d' ' -f2)
         #如果返回http 状代码为204，说明保存到vault成功
        if [[ ${result} == "204" ]]; then
            echo "==> Key/Value successfully written in Vault, done!"
        else
            echo "==> Error while writing Key/Value in Vault!"
            _EXIT="1"
        fi
    ;;
    *)
        echo "==> Unattended Error while reading Key/Value"
        _EXIT="1"
    ;;
esac

# Handle the error exit use case
if [[ ${_EXIT} == "1" ]]; then
    echo "==> Vault request response:"
    cat ${_TMP}
    echo ">>"
fi

echo ""

houseKeeping # temp files and payloads

exit ${_EXIT}

#EOF

 
 

2.3 edgexfoundry/docker-edgex-proxy-go镜像

首先，将security-api-gateway项目源码拷到ubuntn服务器edgexsecurity  目录下，如下结构：

1. myEdgex@instance-nbpv5z80:~/gopath/src$ ls
2. edgexsecurity  github.com
3.  
4. myEdgex@instance-nbpv5z80:~/gopath/src/edgexsecurity$ ls
5. build.bat  CONTRIBUTING.md  Docker      Dockerfile.orgin  glide.yaml  Makefile   vendor   version.go
6. build.sh   core             Dockerfile  glide.lock        LICENSE     README.md  VERSION

复制代码

然后执行如下命令，安装go依赖，vendor会存放起来以备后用

1. glide install

复制代码

2.3.1  Dockerfile.gobuildinside文件

我们采用在镜像内编译的方法，所以解读一下这个文件，此文件我作了修改，内容如下：



#第一阶段，编译为可执行：edgexproxy
#golang:1.9-alpine作为基础镜像，即带有golang的alpine linux操作系统

FROM golang:1.9-alpine AS builder

#待编译的源码务必要放在/go/src下，否由vendor不起作用
RUN mkdir -p /go/src/edgexsecurity

WORKDIR /go/src/edgexsecurity

COPY . .

#这样会很慢，会访问网络，屏蔽了如下两行

#RUN  apk add make
#RUN go get github.com/dghubble/sling && go get github.com/BurntSushi/toml && go get github.com/edgexfoundry/edgex-go/pkg/clients/logging && go get github.com/dgrijalva/jwt-go


#务必要以CGO_ENABLED=0方式编译，否则会报错：standard_init_linux.go:190: exec user process caused "no such file or directory"
RUN cd core && CGO_ENABLED=0 go build -o edgexproxy  #编译好的edgexproxy可执行文件放在core目录下

COPY Docker/res/configuration.toml core/res/

#第二阶段，制作真正的镜像，从第一阶段中拷文件
FROM scratch

LABEL license='SPDX-License-Identifier: Apache-2.0' \
      copyright='Copyright (c) 2018: Dell, Cavium'

#edgexproxy 必须放在与res同级目录，否则找不到configuration.toml，因为main.go文件中这样定义相对目录configFileLocation := flag.String("configfile", "res/configuration.toml", "configuration file")
#除非你在ENTRYPOINT 中指定configfile
WORKDIR /edgexsecurity/core
COPY --from=builder /go/src/edgexsecurity/core/edgexproxy /edgexsecurity/core/
COPY --from=builder /go/src/edgexsecurity/core/res/configuration.toml /edgexsecurity/core/res/configuration.toml

ENTRYPOINT ["./edgexproxy","--init=true"]

至此，镜像文件已制作完毕！


2.3.2  security-api-gateway (简称edgexproxy) 是如何工作的及使用？
请参考：security-api-gateway 模块详解