5.供应链安全

供应链安全

主要内容

❖ 可信任软件供应链概述

❖ 构建镜像Dockerfile文件优化

❖ 镜像漏洞扫描工具:Trivy

❖ 检查YAML文件安全配置:kubesec

❖ 准入控制器: Admission Webhook

❖ 准入控制器: ImagePolicyWebhook

可信任软件供应链概述

可信任软件供应链:指在建设基础架构过程中,涉及的软件都是可信任的。

5.供应链安全_第1张图片

构建镜像Dockerfile文件优化

  • **减少镜像层:**一次RUN指令形成新的一层,尽量Shell命令都写在一行,减少镜像层。
  • **清理无用文件:**清理对应的残留数据,例如yum缓存。
  • **清理无用的软件包:**基础镜像默认会带一些debug工具,可以删除掉,仅保留应用程序所需软件,防止黑客利用。
  • **选择最小的基础镜像:**例如alpine
  • **使用非root用户运行:**USER指令指定普通用户

示例:构建python web镜像

参数化构建如下:

  1. 使用了非root用户运行镜像
  2. 使用了一个镜像层,并非多层
FROM python
RUN useradd python
RUN mkdir /data/www -p
COPY . /data/www
RUN chown -R python /data
RUN pip install flask -i https://mirrors.aliyun.com/pypi/simple/ && yum makecache 
WORKDIR /data/www
USER python
CMD python main.py 

镜像漏洞扫描工具:Trivy

5.供应链安全_第2张图片

Trivy:是一种用于容器镜像、文件系统、Git仓库的漏洞扫描工具。发现目标软件存在的漏洞。 Trivy易于使用,只需安装二进制文件即可进行扫描,方便集成CI系统。

项目地址:https://github.com/aquasecurity/trivy

安装Trivy

在官网下载二进制的文件,移到/usr/local/bin/

[root@master01:~]# mv trivy /usr/local/bin 
[root@master01:~]# trivy --version 
Version: 0.18.3
Vulnerability DB:
  Type: Light
  Version: 1
  UpdatedAt: 2022-04-14 00:51:13.721973471 +0000 UTC
  NextUpdate: 2022-04-14 06:51:13.721972971 +0000 UTC
  DownloadedAt: 0001-01-01 00:00:00 +0000 UTC

下载数据库超时解决

安全扫描通常都需要下载安全漏洞的下载数据库比较完整的版本。数据库速度很慢而且从github下载。所以需要离线。

GitHub项目地址: https://github.com/aquasecurity/trivy-db/releases

下载 trivy-offline.db.tgz 放在trivy cache目录。默认的cache目录的位置上

目录: /home/.cache/trivy/db

[root@master01:~]# cp  trivy-offline.db.tgz .cache/trivy/db/ 
[root@master01:~]# cd .cache/trivy/db/ 
[root@master01:~/.cache/trivy/db] # tar zxvf trivy-offline.db.tgz && rm -rf trivy-offline.db.tgz

Trivy使用

1.容器镜像扫描

也可以指定tar使用trivy image -i nginx.tar

[root@master01:~]# trivy image nginx

+---------------------+------------------+----------+--------------------+-------------------------+-----------------------------------------+
| zlib1g              | CVE-2018-25032   | HIGH     | 1:1.2.11.dfsg-2    | 1:1.2.11.dfsg-2+deb11u1 | zlib: A flaw found in                   |
|                     |                  |          |                    |                         | zlib v1.2.2.2 through zlib              |
|                     |                  |          |                    |                         | v1.2.11 when compressing...             |
|                     |                  |          |                    |                         | -->avd.aquasec.com/nvd/cve-2018-25032   |
+---------------------+------------------+----------+--------------------+-------------------------+-----------------------------------------+

2. 打印指定(高危、严重)漏洞信息

可以通过-s 参数指定是高危漏洞还是严重漏洞

HIGH 是高危漏洞

CRITICAL 是严重漏洞

[root@master01:~] # trivy image -s HIGH,CRITICAL nginx 
2022-04-14T13:38:38.566+0800    INFO    Detected OS: debian
2022-04-14T13:38:38.567+0800    INFO    Detecting Debian vulnerabilities...
2022-04-14T13:38:38.579+0800    INFO    Number of PL dependency files: 1
2022-04-14T13:38:38.579+0800    INFO    Detecting jar vulnerabilities...

nginx (debian 11.2)
===================
Total: 76 (HIGH: 57, CRITICAL: 19)

+---------------------+------------------+----------+--------------------+-------------------------+-----------------------------------------+
|       LIBRARY       | VULNERABILITY ID | SEVERITY | INSTALLED VERSION  |      FIXED VERSION      |                  TITLE                  |
+---------------------+------------------+----------+--------------------+-------------------------+-----------------------------------------+
| curl                | CVE-2021-22945   | CRITICAL | 7.74.0-1.3+deb11u1 |                         | curl: use-after-free and                |
|                     |                  |          |                    |                         | double-free in MQTT sending             |
|                     |                  |          |                    |                         | -->avd.aquasec.com/nvd/cve-2021-22945   |
+                     +------------------+----------+                    +-------------------------+-----------------------------------------+
|                     | CVE-2021-22946   | HIGH     |                    |                         | curl: Requirement to use                |
|                     |                  |          |                    |                         | TLS not properly enforced               |
|                     |                  |          |                    |                         | for IMAP, POP3, and...                  |
|                     |                  |          |                    |                         | -->avd.aquasec.com/nvd/cve-2021-22946   |
+---------------------+------------------+          +--------------------+-------------------------+-----------------------------------------+
| e2fsprogs           | CVE-2022-1304    |          | 1.46.2-2           |                         | e2fsprogs: out-of-bounds                |
|                     |                  |          |                    |                         | read/write via crafted filesystem       |
|                     |                  |          |                    |                         | -->avd.aquasec.com/nvd/cve-2022-1304    |
+---------------------+------------------+          +--------------------+-------------------------+-----------------------------------------+
| gzip                | CVE-2022-1271    |          | 1.10-4             |                         | gzip: arbitrary-file-write              |
|                     |                  |          |                    |                         | vulnerability                           |
|                     |                  |          |                    |                         | -->avd.aquasec.com/nvd/cve-2022-1271    |
+---------------------+------------------+----------+--------------------+-------------------------+-----------------------------------------+

检查YAML文件安全配置:kubesec

kubesec简介

5.供应链安全_第3张图片

kubesec:是一个针对K8s资源清单文件进行安全配置评估的工具,根据安全配置 最佳实践来验证并给出建议。

官网:https://kubesec.io

项目地址:https://github.com/controlplaneio/kubesec

kubesec安装

[root@master01:~]# tar zxvf kubesec_linux_amd64.tar.gz 
CHANGELOG.md
LICENSE
README.md
kubesec
[root@master01:~]# mv kubesec /usr/local/bin/

kubesec使用

示例:

  1. 会有提示告诉你是否使用资源限制
  2. 可以根据建议添加要求去加固Pod的安全范围
$ kubesec scan deployment.yaml 
···
        {
          "id": "RequestsMemory",
          "selector": "containers[] .resources .limits .memory",
          "reason": "Enforcing memory limits prevents DOS via resource exhaustion",
          "points": 1
        },
        {
          "id": "RequestsCPU",
          "selector": "containers[] .resources .requests .cpu",
          "reason": "Enforcing CPU requests aids a fair balancing of resources across the cluster",
          "points": 1
        },

或者使用容器环境执行检查

$ docker run -i kubesec/kubesec scan /dev/stdin < deployment.yaml

kubesec内置一个HTTP服务器,可以直接启用,远程调用。

二进制

kubesec http 8080 &

5.供应链安全_第4张图片

Docker容器

docker run -d -p 8080:8080 kubesec/kubesec http 8080

示例:

curl -sSX POST --data-binary @deployment.yaml http://192.168.31.71:8080/scan

Admission Webhook

Admission Webhook:准入控制器Webhook是准入控制插件的一种,用于拦截所有向APISERVER发送的 请求,并且可以修改请求或拒绝请求。

Admission webhook为开发者提供了非常灵活的插件模式,在kubernetes资源持久化之前,管理员通过程序 可以对指定资源做校验、修改等操作。例如为资源自动打标签、pod设置默认SA,自动注入sidecar容器等。

相关Webhook准入控制器:

  • MutatingAdmissionWebhook:修改资源,理论上可以监听并修改任何经过ApiServer处理的请求
  • ValidatingAdmissionWebhook:验证资源
  • ImagePolicyWebhook:镜像策略,主要验证镜像字段是否满足条件

5.供应链安全_第5张图片

ImagePolicyWebhook

镜像策略的工作流程

5.供应链安全_第6张图片

启用准入控制器插件

  1. 提供HTTPS证书
  2. 添加配置文件
  3. 需要启用准入控制器开启ImagePolicyWebhook插件
1.生成HTTPS证书
  • 需要使用cfssl工具、生成https证书
  • 需要admission_configurationconnect_webhook的配置文件
[root@master01:~/image-webhook]# ll
total 18820
-rw-r--r-- 1 root root      508 1021 18:53 admission_configuration.yaml
-rwxr-xr-x 1 root root 10376657 1124 2019 cfssl
-rwxr-xr-x 1 root root  6595195 1124 2019 cfssl-certinfo
-rwxr-xr-x 1 root root  2277873 1124 2019 cfssljson
-rw-r--r-- 1 root root      632 1021 18:46 connect_webhook.yaml
-rw-r--r-- 1 root root     1367 75 2021 image-policy-certs.sh
drwxr-xr-x 2 root root       39 79 2021 image-policy-webhook


[root@master01:~/image-webhook]# vim image-policy-certs.sh
cat > webhook-csr.json <<EOF
{
  "CN": "webhook",
  "hosts": [
   "10.11.121.111"   # 修改一下IP,我这里是master
   ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}

通过cfssl工具生成https认证

  • 生成的key和pem文件需要复制到imagepolicywebhook需要的目录
  • 还需要将两个连接webhook配置文件复制到指定目录
[root@master01:~/image-webhook]# mv cfssl* /usr/local/bin/
[root@master01:~/image-webhook]# chmod +x image-policy-certs.sh 
[root@master01:~/image-webhook]# ./image-policy-certs.sh 
[root@master01:~/image-webhook]# ll
total 64
-rw-r--r-- 1 root root  508 1021 18:53 admission_configuration.yaml
-rw-r--r-- 1 root root  956 414 09:31 apiserver-client.csr
-rw-r--r-- 1 root root  182 414 09:31 apiserver-client-csr.json
-rw------- 1 root root 1675 414 09:31 apiserver-client-key.pem
-rw-r--r-- 1 root root 1306 414 09:31 apiserver-client.pem
-rw-r--r-- 1 root root  294 414 09:31 ca-config.json
-rw-r--r-- 1 root root  960 414 09:31 ca.csr
-rw-r--r-- 1 root root  212 414 09:31 ca-csr.json
-rw------- 1 root root 1675 414 09:31 ca-key.pem
-rw-r--r-- 1 root root 1273 414 09:31 ca.pem
-rw-r--r-- 1 root root  632 1021 18:46 connect_webhook.yaml
-rwxr-xr-x 1 root root 1367 75 2021 image-policy-certs.sh
drwxr-xr-x 2 root root   39 79 2021 image-policy-webhook
-rw-r--r-- 1 root root 1001 414 09:31 webhook.csr
-rw-r--r-- 1 root root  204 414 09:31 webhook-csr.json
-rw------- 1 root root 1679 414 09:31 webhook-key.pem
-rw-r--r-- 1 root root 1330 414 09:31 webhook.pem
2.添加配置文件
  • 查看admission_configuration.yaml
  • 查看connect_webhook.yaml
  • /etc/kubernetes/image-policy/该目录是不存在的,所以需要创建,用作数据卷持久化
  • 需要修改的是connect_webhook的server地址,该地址是镜像策略服务地址,这里我使用master01
  • admission_configuration.yamlconnect_webhook.yaml需要存放到/etc/kubernetes/image-policy/目录
[root@master01:~/image-webhook] # cat admission_configuration.yaml 
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  configuration:
    imagePolicy:
      kubeConfigFile: /etc/kubernetes/image-policy/connect_webhook.yaml  # 连接镜像策略服务器配置文件
      allowTTL: 50 # 控制批准请求的缓存时间,单位秒
      denyTTL: 50 # 控制批准请求的缓存时间,单位秒
      retryBackoff: 500 # 控制重试间隔,单位毫秒
      defaultAllow: true # 确定webhook后端失效的行为
      
      
[root@master01:~/image-webhook] # cat connect_webhook.yaml 
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/image-policy/webhook.pem # 数字证书,用于验证远程服务
    server: https://10.11.121.111:8080/image_policy # 镜像策略服务器地址,必须是https
  name: webhook
contexts:
- context:
    cluster: webhook
    user: apiserver
  name: webhook
current-context: webhook 
preferences: {}
users:
- name: apiserver
  user:
    client-certificate: /etc/kubernetes/image-policy/apiserver-client.pem # webhook准入控制器使用的证书
    client-key: /etc/kubernetes/image-policy/apiserver-client-key.pem # 对应私钥证书 

新建/etc/kubernetes/image-policy的目录,复制文件到该目录下

[root@master01:~/image-webhook]# mkdir /etc/kubernetes/image-policy
[root@master01:~/image-webhook]# cp admission_configuration.yaml connect_webhook.yaml /etc/kubernetes/image-policy/
[root@master01:~/image-webhook]# cp apiserver*  /etc/kubernetes/image-policy/
[root@master01:~/image-webhook]# cp webhook* /etc/kubernetes/image-policy/
[root@master01:~/image-webhook]# cp ca.pem  /etc/kubernetes/image-policy/
[root@master01:~/image-webhook]# ll /etc/kubernetes/image-policy/
total 44
-rw-r--r-- 1 root root  508 414 09:38 admission_configuration.yaml
-rw-r--r-- 1 root root  956 414 09:39 apiserver-client.csr
-rw-r--r-- 1 root root  182 414 09:39 apiserver-client-csr.json
-rw------- 1 root root 1675 414 09:39 apiserver-client-key.pem
-rw-r--r-- 1 root root 1306 414 09:39 apiserver-client.pem
-rw-r--r-- 1 root root 1273 414 09:39 ca.pem
-rw-r--r-- 1 root root  632 414 09:38 connect_webhook.yaml
-rw-r--r-- 1 root root 1001 414 09:39 webhook.csr
-rw-r--r-- 1 root root  204 414 09:39 webhook-csr.json
-rw------- 1 root root 1679 414 09:39 webhook-key.pem
-rw-r--r-- 1 root root 1330 414 09:39 webhook.pem
3.开启ImagePolicyWebhook插件

使用hostpath数据卷将宿主机/etc/kubernetes/image-policy目录挂载到容器中。

重启kubelet,等待一分钟重建kube-apiserver的Pod。

[root@master01:~/image-webhook]# vim /etc/kubernetes/kube-apiserver.yaml 
 	- --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook
	- --admission-control-config-file=/etc/kubernetes/image-policy/admission_configuration.yaml
···
    volumeMounts:
    - mountPath: /etc/kubernetes/image-policy
      name: image-policy
···
  volumes:
  - hostPath:
      path: /etc/kubernetes/image-policy
      type: Directory
      name: image-policy
      
[root@master01:~/image-webhook]# systemctl  restart  kubelet 
4.部署镜像服务器

在本机的master节点部署镜像服务器,来当server,要求如下:

  1. 使用Python编写一个连接ImagePolicyWebhook的api接口
  2. 编写一个Dockerfile运行这个python文件
[root@master01:~/image-webhook]# mv webhook* image-policy-webhook/
[root@master01:~/image-webhook/image-policy-webhook] # ll
total 24
-rw-r--r-- 1 root root  212 79 2021 Dockerfile
-rw-r--r-- 1 root root  982 79 2021 main.py
-rw-r--r-- 1 root root 1001 414 09:31 webhook.csr
-rw-r--r-- 1 root root  204 414 09:31 webhook-csr.json
-rw------- 1 root root 1679 414 09:31 webhook-key.pem
-rw-r--r-- 1 root root 1330 414 09:31 webhook.pem

查看当前的Dockerfile文件

[root@master01:~/image-webhook/image-policy-webhook] # cat Dockerfile 
FROM python
RUN useradd python
RUN mkdir /data/www -p
COPY . /data/www
RUN chown -R python /data
RUN pip install flask -i https://mirrors.aliyun.com/pypi/simple/
WORKDIR /data/www
USER python
CMD python main.py 

查看当前的Python脚本

[root@master01:~/image-webhook/image-policy-webhook] # cat main.py 
from flask import Flask,request
import json

app = Flask(__name__)

@app.route('/image_policy',methods=["POST"])
def image_policy():
    post_data = request.get_data().decode()
    #print("POST数据: %s" %post_data)
    data = json.loads(post_data)
    for c in data['spec']['containers']:
        # 如果镜像里不带冒号或者带:latest说明是镜像使用latest标签
        if ":" not in c['image'] or ":latest" in c['image']:  
            allowed, reason = False, "检查镜像失败!镜像标签不允许使用latest!"
            break
        else:
            allowed, reason = True, "检查镜像通过."
    print("检查结果: %s" %reason)
    result = {"apiVersion": "imagepolicy.k8s.io/v1alpha1","kind": "ImageReview",
              "status": {"allowed": allowed,"reason": reason}}

    return json.dumps(result,ensure_ascii=False) 

if __name__ == "__main__":
    app.run(host="0.0.0.0",port=8080,ssl_context=('/data/www/webhook.pem','/data/www/webhook-key.pem'))

构建Dockerfile

[root@master01:~/image-webhook/image-policy-webhook]# docker build -t image-policy-webhook .
Digest: sha256:dbbfcbf95f6b596d2be1d8f3b368016619f78f829facf6f2e361bea1151794e5
Status: Downloaded newer image for python:latest
 ---> a5d7930b60cc
Step 2/9 : RUN useradd python
 ---> Running in fcd9fcc7e44f
Removing intermediate container fcd9fcc7e44f
 ---> 10b842f26fed
Step 3/9 : RUN mkdir /data/www -p
 ---> Running in cffd0d6f3bbd
Removing intermediate container cffd0d6f3bbd
 ---> 4db5fe4dc849
Step 4/9 : COPY . /data/www
 ---> d27fa5502133
Step 5/9 : RUN chown -R python /data
 ---> Running in 7559149b085b
Removing intermediate container 7559149b085b
 ---> a35d92963cfd
Step 6/9 : RUN pip install flask -i https://mirrors.aliyun.com/pypi/simple/
 ---> Running in 8aea1cb3d0fe
 Removing intermediate container 8aea1cb3d0fe
 ---> b81eed200f98
Step 7/9 : WORKDIR /data/www
 ---> Running in fb82b0cc633e
Removing intermediate container fb82b0cc633e
 ---> 2e43f0dff4ad
Step 8/9 : USER python
 ---> Running in 5faeb5a8566e
Removing intermediate container 5faeb5a8566e
 ---> 6797205b1622
Step 9/9 : CMD python main.py
 ---> Running in da4ae6eabc00
Removing intermediate container da4ae6eabc00
 ---> d49ca131fdb9
Successfully built d49ca131fdb9
Successfully tagged image-policy-webhook:latest
5.部署镜像服务

自己用python开发一个简单的webhook端点服务器,作用是拒绝部署的镜像乜有指定标签(即latest)。

  1. 自签HTTPS证书
  2. Docker容器启动镜像策略服务
  3. 查看当前的Docker日志
[root@master01:~/image-webhook/image-policy-webhook]# docker run -d -u root --name=image-policy-webhook \
> -v $PWD/webhook.pem:/data/www/webhook.pem \
> -v $PWD/webhook-key.pem:/data/www/webhook-key.pem \
> -e PYTHONUNBUFFERED=1 -p 8080:8080 \
> image-policy-webhook
73a9ae72a7bc8f379fe5414f575b779bf34c969089f6e544e1c7a81a60fc5b9d.


[root@master01:~/image-webhook/image-policy-webhook]# docker logs -f image-policy-webhook 
 * Serving Flask app 'main' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on https://127.0.0.1:8080
 * Running on https://172.17.0.2:8080 (Press CTRL+C to quit)

测试ImagePolicyWebhook

创建Deployment使用nginx镜像,使用nginx:1.16的版本,查看日志。

[root@master01 ~]# kubectl create deployment web1 --image=nginx:1.16 
deployment.apps/web1 created

[root@master01 ~]# docker logs  -f image-policy-webhook 
 * Serving Flask app 'main' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on https://127.0.0.1:8080
 * Running on https://172.17.0.2:8080 (Press CTRL+C to quit)
POST数据: {"kind":"ImageReview","apiVersion":"imagepolicy.k8s.io/v1alpha1","metadata":{"creationTimestamp":null},"spec":{"containers":[{"image":"nginx:1.16"}],"namespace":"default"},"status":{"allowed":false}}

检查结果: 检查镜像通过.
10.11.121.118 - - [14/Apr/2022 14:00:46] "POST /image_policy?timeout=30s HTTP/1.1" 200 -
POST数据: {"kind":"ImageReview","apiVersion":"imagepolicy.k8s.io/v1alpha1","metadata":{"creationTimestamp":null},"spec":{"containers":[{"image":"nginx:latest"}],"namespace":"default"},"status":{"allowed":false}}

创建Deployment使用nginx镜像,使用nginx:latest的版本,查看日志。

[root@master01 ~]# kubectl create deployment web2 --image=nginx:latest
deployment.apps/web2 created

[root@master01 image-policy-webhook]# docker logs  -f image-policy-webhook 
 * Serving Flask app 'main' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses (0.0.0.0)
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on https://127.0.0.1:8080
 * Running on https://172.17.0.2:8080 (Press CTRL+C to quit)
POST数据: {"kind":"ImageReview","apiVersion":"imagepolicy.k8s.io/v1alpha1","metadata":{"creationTimestamp":null},"spec":{"containers":[{"image":"nginx:1.16"}],"namespace":"default"},"status":{"allowed":false}}

检查结果: 检查镜像通过.
10.11.121.118 - - [14/Apr/2022 14:00:46] "POST /image_policy?timeout=30s HTTP/1.1" 200 -
POST数据: {"kind":"ImageReview","apiVersion":"imagepolicy.k8s.io/v1alpha1","metadata":{"creationTimestamp":null},"spec":{"containers":[{"image":"nginx:latest"}],"namespace":"default"},"status":{"allowed":false}}

检查结果: 检查镜像失败!镜像标签不允许使用latest!
10.11.121.118 - - [14/Apr/2022 14:01:04] "POST /image_policy?timeout=30s HTTP/1.1" 200 -
POST数据: {"kind":"ImageReview","apiVersion":"imagepolicy.k8s.io/v1alpha1","metadata":{"creationTimestamp":null},"spec":{"containers":[{"image":"nginx:latest"}],"namespace":"default"},"status":{"allowed":false}}

检查结果: 检查镜像失败!镜像标签不允许使用latest!
10.11.121.118 - - [14/Apr/2022 14:02:26] "POST /image_policy?timeout=30s HTTP/1.1" 200 -

你可能感兴趣的:(CKS,安全,python,flask,kubernetes)