K8S部署Harbor仓库实战

Docker容器应用的开发和运行离不开可靠的镜像管理，虽然Docker官方也提供了公共的镜像仓库，包括阿里云、腾讯云等都有镜像仓库，但是总是有各自的限制，出于安全和效率等方面考虑，部署私有环境内的Registry也是非常必要的。Harbor是由VMware公司开源的企业级的Docker Registry管理项目，它包括权限管理(RBAC)、LDAP、日志审核、管理界面、自我注册、镜像复制和中文支持等功能。在K8S中安装Harbor的过程如下：

环境准备

系统版本：ubuntu 20.04
Kubernetes版本：v1.21.5
Helm版本：v3.6.3
Chart版本：1.8.1

安装流程

准备`helm`仓库

添加仓库

helm repo add harbor https://helm.goharbor.io

更新仓库

helm repo update

准备`namespace`

kubectl create namespace public

准备持久化目录

持久化方面，这里采用NFS来做持久化存储。另外，请给harbor目录足够的权限，否则redis和database会报错，我这里直接给了harbor 777权限

chmod -R 777 /mydata/k8s/public/harbor

NFS Server: 192.168.5.22
chartmuseum目录: /mydata/k8s/public/harbor/chartmuseum
database目录: /mydata/k8s/public/harbor/database
jobservice目录: /mydata/k8s/public/harbor/jobservice
redis目录: /mydata/k8s/public/harbor/redis
registry目录: /mydata/k8s/public/harbor/registry
trivy目录: /mydata/k8s/public/harbor/trivy

准备PV

资源文件harbor-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-registry
  labels:
    app: harbor-registry
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/registry
    server: 192.168.5.22
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-chartmuseum
  labels:
    app: harbor-chartmuseum
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/chartmuseum
    server: 192.168.5.22
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-jobservice
  labels:
    app: harbor-jobservice
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/jobservice
    server: 192.168.5.22
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-database
  labels:
    app: harbor-database
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/database
    server: 192.168.5.22
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-redis
  labels:
    app: harbor-redis
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/redis
    server: 192.168.5.22
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: harbor-trivy
  labels:
    app: harbor-trivy
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "nfs-client"
  mountOptions:
    - hard
  nfs:
    path: /mydata/k8s/public/harbor/trivy
    server: 192.168.5.22

创建PV

kubectl apply -f pv.yaml

准备PVC

资源文件harbor-pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-registry
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 50Gi
  selector:
    matchLabels:
      app: harbor-registry
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-chartmuseum
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      app: harbor-chartmuseum
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-jobservice
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      app: harbor-jobservice 
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-database
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      app: harbor-database  
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-redis
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      app: harbor-redis
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: harbor-trivy
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: "nfs-client"
  resources:
    requests:
      storage: 5Gi
  selector:
    matchLabels:
      app: harbor-trivy

创建pvc

kubectl apply -f harbor-pvc.yaml -n public

准备配置清单

资源文件harbor-values.yaml

这里的192.168.4.224，需要替换为自己的节点地址

expose:
  type: ingress
  tls:
    enabled: true
  clusterIP:
    name: harbor
    annotations: {}
    ports:
      httpPort: 80
      httpsPort: 443
      notaryPort: 4443
  ingress:
    hosts:
      core: harbor-core.public.192.168.4.224.nip.io
      notary: harbor-notary.public.192.168.4.224.nip.io
    controller: default
    kubeVersionOverride: ""
    annotations:
      ingress.kubernetes.io/ssl-redirect: "true"
      ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
    notary:
      annotations: {}
    harbor:
      annotations: {}
externalURL: https://harbor-core.public.192.168.4.224.nip.io:31839

persistence:
  enabled: true
  resourcePolicy: "keep"
  persistentVolumeClaim:
    registry:
      existingClaim: "harbor-registry"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 50Gi
    chartmuseum:
      existingClaim: "harbor-chartmuseum"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 5Gi
    jobservice:
      existingClaim: "harbor-jobservice"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 5Gi
    database:
      existingClaim: "harbor-database"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 5Gi
    redis:
      existingClaim: "harbor-redis"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 5Gi
    trivy:
      existingClaim: "harbor-trivy"
      storageClass: "nfs-client"
      subPath: ""
      accessMode: ReadWriteOnce
      size: 5Gi

安装Harbor

helm install harbor harbor/harbor -f harbor-values.yaml -n public

访问Harbor

用户名: admin
密码：Harbor12345
https://harbor-core.cloud-platform-public.192.168.4.224.nip.io:31839

修复`docker login`报错

修改docker配置文件，添加以下字段

"insecure-registries": ["harbor-core.public.192.168.4.224.nip.io:31839"]

现在就可以登录并推送镜像了。