k8s 安装harbor
前言
文章篇幅较长，请详细阅读，不要漏掉每一个注释，每一个细节。
文章中用到的harbor存储是hostpath，service的类型是ingress。根据harbor官网提示，需要提前创建pv和pvc，既然选择了hostpath，那么就需要将pod固定到某一个node上面，本文所有的资源副本都是1，harbor的所有pod放在同一个namespace，并且需要将namespace固定到某个node上，具体步骤见正文。
一.环境准备
2.创建namespace
cat namespace-harbor.yaml
apiVersion: v1
kind: Namespace
metadata:
name: harborkubectl apply -f namespace-harbor.yaml
3.开启准入控制器
如果是多master，所有master都要修改
–enable-admission-plugins 添加 PodNodeSelector
vim /etc/kubernetes/manifests/kube-apiserver.yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.0.20:6443
creationTimestamp: null
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver
namespace: kube-system
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=192.168.0.20
- --allow-privileged=true
- --authorization-mode=Node,RBAC
- --client-ca-file=/etc/kubernetes/pki/ca.crt
- --enable-admission-plugins=NodeRestriction,PodNodeSelector
apiserver pod会自动重启
4.命名空间加注解
kubectl edit ns harbor -o yaml
apiVersion: v1
kind: Namespace
metadata:
annotations:
scheduler.alpha.kubernetes.io/node-selector: harbor=env
name: harbor
5.node打标签
kubectl label node k8s-node3 harbor=env
6.创建pv和pvc
选择hostpath，需要提前准备pv和pvc，官网解释如下：
官网链接：https://goharbor.io/docs/2.7.0/install-config/harbor-ha-helm/
在k8s-node3节点创建目录，建议给/data下单独挂一块存储盘
df -h
mkdir -pv /data/harbor/{chartmuseum,database,jobservice,redis,registry,scandata,trivy}
mkdir: created directory ‘harbor’
mkdir: created directory ‘harbor/chartmuseum’
mkdir: created directory ‘harbor/database’
mkdir: created directory ‘harbor/jobservice’
mkdir: created directory ‘harbor/redis’
mkdir: created directory ‘harbor/registry’
mkdir: created directory ‘harbor/scandata’
mkdir: created directory ‘harbor/trivy’
chmod 777 -R /data/harbor ##必须要赋权，不然pod启动会有问题
创建pv和pvc
cat harbor-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-registry-pv"
labels:
name: harbor-registry-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/registry
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-chartmuseum-pv"
labels:
name: harbor-chartmuseum-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/chartmuseum
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-jobservice-pv"
labels:
name: harbor-jobservice-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/jobservice
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-database-pv"
labels:
name: harbor-database-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/database
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-redis-pv"
labels:
name: harbor-redis-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/redis
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-trivy-pv"
labels:
name: harbor-trivy-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/trivy
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: "harbor-scandata-pv"
labels:
name: harbor-scandata-pv
release: stable
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /data/harbor/scandata
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-registry-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-registry-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-chartmuseum-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-chartmuseum-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-jobservice-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-jobservice-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-database-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-database-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-redis-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-redis-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-trivy-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-trivy-pv
release: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: harbor-scandata-pvc
namespace: harbor
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
selector:
matchLabels:
name: harbor-scandata-pv
release: stable
kubectl apply -f harbor-pv.yaml
kubectl get pv,pvc -n harborNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/harbor-chartmuseum-pv 5Gi RWO Retain Bound harbor/harbor-chartmuseum-pvc 31h
persistentvolume/harbor-database-pv 5Gi RWO Retain Bound harbor/harbor-database-pvc 31h
persistentvolume/harbor-jobservice-pv 5Gi RWO Retain Bound harbor/harbor-jobservice-pvc 31h
persistentvolume/harbor-redis-pv 5Gi RWO Retain Bound harbor/harbor-redis-pvc 31h
persistentvolume/harbor-registry-pv 5Gi RWO Retain Bound harbor/harbor-registry-pvc 31h
persistentvolume/harbor-scandata-pv 5Gi RWO Retain Bound harbor/harbor-scandata-pvc 30h
persistentvolume/harbor-trivy-pv 5Gi RWO Retain Bound harbor/harbor-trivy-pvc 31h
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/harbor-chartmuseum-pvc Bound harbor-chartmuseum-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-database-pvc Bound harbor-database-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-jobservice-pvc Bound harbor-jobservice-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-redis-pvc Bound harbor-redis-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-registry-pvc Bound harbor-registry-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-scandata-pvc Bound harbor-scandata-pv 5Gi RWO 30h
persistentvolumeclaim/harbor-trivy-pvc Bound harbor-trivy-pv 5Gi RWO 31h
二.安装harbor
1.添加harbor的helm源
helm repo add harbor https://helm.goharbor.io
helm search repo
helm pull harbor/harbor
tar zxf harbor-1.11.0.tgz
cd harbor/
lscert Chart.yaml conf LICENSE README.md templates values.yaml
2.前方高能，修改values.yaml
这里用默认类型 ingress，https协议，如果想用http协议，那么需要删除掉“expose.ingress.annotations”里面的“ssl-redirect”相关注解
vim values.yaml
expose:
# Set how to expose the service. Set the type as "ingress", "clusterIP", "nodePort" or "loadBalancer"
# and fill the information in the corresponding section
type: ingress
tls:
# Enable TLS or not.
# Delete the "ssl-redirect" annotations in "expose.ingress.annotations" when TLS is disabled and "expose.type" is "ingress"
# Note: if the "expose.type" is "ingress" and TLS is disabled,
# the port must be included in the command when pulling/pushing images.
# Refer to https://github.com/goharbor/harbor/issues/5291 for details.
enabled: true
# The source of the tls certificate. Set as "auto", "secret"
# or "none" and fill the information in the corresponding section
# 1) auto: generate the tls certificate automatically
# 2) secret: read the tls certificate from the specified secret.
# The tls certificate can be generated manually or by cert manager
# 3) none: configure no tls certificate for the ingress. If the default
# tls certificate is configured in the ingress controller, choose this option
certSource: auto
修改持久化配置，将pvc name添加在existingClaim后面
persistence:
enabled: true
# Setting it to "keep" to avoid removing PVCs during a helm delete
# operation. Leaving it empty will delete PVCs after the chart deleted
# (this does not apply for PVCs that are created for internal database
# and redis components, i.e. they are never deleted automatically)
resourcePolicy: "keep"
persistentVolumeClaim:
registry:
# Use the existing PVC which must be created manually before bound,
# and specify the "subPath" if the PVC is shared with other components
existingClaim: "harbor-registry-pvc"
# Specify the "storageClass" used to provision the volume. Or the default
# StorageClass will be used (the default).
# Set it to "-" to disable dynamic provisioning
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
chartmuseum:
existingClaim: "harbor-chartmuseum-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
jobservice:
jobLog:
existingClaim: "harbor-jobservice-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
scanDataExports:
existingClaim: "harbor-scandata-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
# If external database is used, the following settings for database will
# be ignored
database:
existingClaim: "harbor-database-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
# If external Redis is used, the following settings for Redis will
# be ignored
redis:
existingClaim: "harbor-redis-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
trivy:
existingClaim: "harbor-trivy-pvc"
storageClass: ""
subPath: ""
accessMode: ReadWriteOnce
size: 5Gi
annotations: {}
3.helm安装harbor
helm install harbor . -f values.yaml -n harbor删除命令
helm uninstall harbor -n harbor
kubectl get pod -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-chartmuseum-7b74f8b585-qcp89 1/1 Running 2 4h39m 10.244.107.231 k8s-node3
harbor-core-7fc48678c8-qcdqk 1/1 Running 2 4h39m 10.244.107.233 k8s-node3
harbor-database-0 1/1 Running 2 4h39m 10.244.107.236 k8s-node3
harbor-jobservice-8486bb4bcb-2gjtp 1/1 Running 9 4h39m 10.244.107.237 k8s-node3
harbor-notary-server-7f7bf8f6d-zphpg 1/1 Running 2 4h39m 10.244.107.244 k8s-node3
harbor-notary-signer-5f9df848b7-skpxj 1/1 Running 2 4h39m 10.244.107.248 k8s-node3
harbor-portal-748c6db9c6-qw97j 1/1 Running 2 4h39m 10.244.107.243 k8s-node3
harbor-redis-0 1/1 Running 2 4h39m 10.244.107.228 k8s-node3
harbor-registry-6777c99d8d-cb9rj 2/2 Running 4 4h39m 10.244.107.235 k8s-node3
harbor-trivy-0 1/1 Running 2 4h39m 10.244.107.245 k8s-node3
4.ingress绑定ingress class
kubectl get ingress -n harbor
NAME CLASS HOSTS ADDRESS PORTS AGE
harbor-ingress
kubectl edit ingress harbor-ingress -n harbor
......
spec:
ingressClassName: nginx ## k8s-1.20.X添加ingressClassName
rules:
- host: core.harbor.domain
http:
paths:
......
[root@k8s-master harbor]# kubectl get ingress -n harbor ##查看添加成功
NAME CLASS HOSTS ADDRESS PORTS AGE
harbor-ingress nginx core.harbor.domain 192.168.0.21,192.168.0.22 80, 443 4h57m
5.服务器配置证书，不然docker login和docker pull 都会报错
在harbor-ingress 的secret里面找到data下的ca.crt
kubectl get secret harbor-ingress -n harbor -o yaml
apiVersion: v1
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRY3d4c0NtVmNPV2RpZWJTSVREcU9kakFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05Nakl4TWpJNE1EWXdOVE13V2hjTk1qTXhNakk0TURZdwpOVE13V2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRQ3g0YVBFbkJ5K2xicElsSytyZmlsdndIWkJRMzM1a3lxMThKWHhpcDl1SFQ0VkExUzEKbG45anVtQUR4Smk4QnhWZStmVlBDWW5UeGJoNWJua3pqdm9XekptOTlMRUY5UDJPdW5VZnZCWGt6VFZEK0o1TgpBZjhyMmtqMFRTck5yZVVKL3JwR01mT1V1MFRSSjg2N2g1WHg1aEpBeDNlWlk4LzRFWmZQOVFGRlFsVGRxRDFBCmlkUkRydjFCSGJIajNxVjU3ckI4L3VxMVJ1a2hjQUZodlZXKzMvdmNkelBaS0hjc0c4d3VhNml2ekExK096a1gKTFA2K2wvcnlPektWWG9kU3ZHV1RmVzJpSlFEUjJqSWliR1loSUF3SDIrZlU4MVJrdGNsS050b2hZYW5aYjdOdQpTcTlzOFB1b3Y2V1JDMWpwRUd6cjhtR2hKTEhtcHJMNlRXeDdBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVZc2xqdmpDZ3F5R2NRZ1ZWS0s5TnIreU8yWll3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFKd1NkMkFuMmpEdHp6NUIvOEl2UzliamxVN3hLVlR0M1pWWkc0N0R5aFJXTmRpVEs1NFF3dFdhCmtCYUZrV2xTQWJaUk5iOWwyak85NEFMelRCb0ttVU04QTBlZFBQbXdvTFFaZlV2NVp6SCtGM05PenNVT1I5dFIKbnhva3lkSU14WUlpQWp1YThudXRLK0g2TklsRDRiVVA1MVZSNFJsNjBhdHhSWi81OW1HMWw1ZUNVSWtObVJyaQpwY2FQTjBBa3NzR2dCNmpxK3VXMXhPU0l1M21idVl3amdzTEpqWTFkNkNuS1IvQ1RRZnRBKzB2NFNVRTJBTFRGClg0TEFRVXFsM0F2NU8wWkxFQXAzNE5aR0xGNCtnRmJuYi84MmdYTlBuY2IzQkxOR0tKVE5zL0RGTTBkYlgwajcKY3VIMHQvT01oV2YyZ084eTNBMDB6dkIzcTVZSkIwdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURVekNDQWp1Z0F3SUJBZ0lSQU9QeW44V1FlOVZOaUlYL21LSGhrSHN3RFFZSktvWklodmNOQVFFTEJRQXcKRkRFU01CQUdBMVVFQXhNSmFHRnlZbTl5TFdOaE1CNFhEVEl5TVRJeU9EQTJNRFV6TUZvWERUSXpNVEl5T0RBMgpNRFV6TUZvd0hURWJNQmtHQTFVRUF4TVNZMjl5WlM1b1lYSmliM0l1Wkc5dFlXbHVNSUlCSWpBTkJna3Foa2lHCjl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2dno0N0l4d2JUZDBQOFcvWngxUTVYaS9IRWZLajJmRzFlK08KQ0RMb2psSWEvbmErUlpaRzZvSVNIbGt6TVo2cTh3WmhZeTRVRTFCQi9VM1hKK2xuNHNhTkgrVllTQjdDbmFuYQpiWW5DVkp2Y091d1J3LzVaNUJzU1UreE55U0tLd3dGN0hLcDZBdklIc1puaU85aHhMNUVKRjRmbmtHY3BQZlZKCjNQTzdMY3B4ZDBVcEJ0enJyYnBWWTV1eDlmY3ZKSWNFeEJTY3ZmaGxmN3VjR0pYSDBZa0xra3RJUVAveWdRekMKQWMvUG82NDJ6VUJqNE1RWGVaRVQ2WjRxaWtLWnc4VXRGTyttZmZBU3BPT3FmMkpncDJiaU4yR1VqR044MTkxTAo1NU1BUUY3aFBzUWo0Y0k2aEVWczk3VmtXci9ZKzdxN3EyNkZnQ1IxY0p6ZVBQY1puUUlEQVFBQm80R1dNSUdUCk1BNEdBMVVkRHdFQi93UUVBd0lGb0RBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFRWUlLd1lCQlFVSEF3SXcKREFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVdnQlJpeVdPK01LQ3JJWnhDQlZVb3IwMnY3STdabGpBegpCZ05WSFJFRUxEQXFnaEpqYjNKbExtaGhjbUp2Y2k1a2IyMWhhVzZDRkc1dmRHRnllUzVvWVhKaWIzSXVaRzl0CllXbHVNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJpK3pFdFllS3A3VnRnamo1Vk9PMXUzRERLSmJFaXZYRFYKbG9ZeVRjQzdWRGg3YlF5UmhPVEtja2pJeGd6cFpubjUyczYvSHYxbkQ3MTk0UXFieE0yOGI1OEpxUEtvS0E4aQpWNHEzSE9xUGhlMlRoS1M4ZUt4anl4R0ZRYi8rQ2prdXNlTDV0bjVFbFM1SjBIRU93K2crZjk0WUdMMTFhdkNuCklnazdDbjQwTWovUUg1YTdhaC9lenNad3Y1aFJISHQrc2pyeE5TYWFTcXNjbHQ4bEV2M3VQWFVsdHBrRm1VY2IKc3hOM1lwRDg4d05LWElQSnU5bWpGMnFJUUdUYXBtME04a3lvTGZpMHlrdDlWM1I1cXoybVcwQWx5d3dob0NvMQpTTm9KbUJIOE42bnhqU1JPUzVJS01XU0RhYWtLc1ZVWGhiUjJRQVlyYVkvZ3YrWnhmZEdlCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdnZ6NDdJeHdiVGQwUDhXL1p4MVE1WGkvSEVmS2oyZkcxZStPQ0RMb2psSWEvbmErClJaWkc2b0lTSGxrek1aNnE4d1poWXk0VUUxQkIvVTNYSitsbjRzYU5IK1ZZU0I3Q25hbmFiWW5DVkp2Y091d1IKdy81WjVCc1NVK3hOeVNLS3d3RjdIS3A2QXZJSHNabmlPOWh4TDVFSkY0Zm5rR2NwUGZWSjNQTzdMY3B4ZDBVcApCdHpycmJwVlk1dXg5ZmN2SkljRXhCU2N2ZmhsZjd1Y0dKWEgwWWtMa2t0SVFQL3lnUXpDQWMvUG82NDJ6VUJqCjRNUVhlWkVUNlo0cWlrS1p3OFV0Rk8rbWZmQVNwT09xZjJKZ3AyYmlOMkdVakdOODE5MUw1NU1BUUY3aFBzUWoKNGNJNmhFVnM5N1ZrV3IvWSs3cTdxMjZGZ0NSMWNKemVQUGNablFJREFRQUJBb0lCQUEyOEVZYXBYdGhyMlg5UQprRUF4R29VMkZzQkk1V2RFdGtiaVVGdUVsYmJPYnNTSDg0ejdqcVFFc0pyZ0hVOWZNcm9CWm5XRWRjc2h4VzBhCjhQTWMxT3k1REtNVGtqSzFpNWRkQktsRjgrU3p4TFcwRGw4QzVxSUIxR0tXcHMxbEg0ZUFqc0x6KzR4RXJZcjMKSHc3Z3ROc1AyaENJSkgzOC9UbEliNFJsUHJ1L0tIV2p1SzJQWWQweFdzdDBtWVZBMytpYW9jWWJXTk1BV3p5cAo0WmR0c1Via2MrU3JPVUIvM2hrNE9iNUtkbUM3RE5aMTZJTnlSNGdpdXliSFI0Wk9JaHh1NGsyenFhVDBoU2J5CjREaUEyNytsczNWcGxETHhmanlKUVQyMEZHVW5jNTllb1pzdzBUeHZSUjM5cEFJQW9IdUhlL0ZCQW1RRnhZSEEKMS9qNUp5RUNnWUVBMnFnVDd4Sms4WFNLcStwenhDSTV4TlpTRVp0OU5WeVFZUUFzM25ubkxhemp0UUUwSGJGeQpWbk9ja1h6ZFd4Wnd5WUtUTktSb0ViSGE3V0NUVmd0T3RrR3VJWCtnenprQTRQaEI1YmhFNmp2bUhzS0laQkkvCmN5Z1lvMENZY0dZTGl5YnpRV0VLTzRPRHg1TlVxZmY2OTVRVVRGek1ORE10eSsvRnozc3lYNVVDZ1lFQTM1c3oKUmZHa0Qzd1lDTSszbWR3VldOZnRjbkNBYmw1YXppM3BWRG4yR2JRNC9GYVZkYVNmT3Z0Vk1oRUZpZkZGS2RzRApSNXJ3SmdXNmJSRUg0V1JOSEQxUzZ2WHc5ZnJJRkZFMnBRa2JZb01iV0tsUlFMNTlSYlRMQnEzQVJnV3Q4MXBYCmJtOXNNeFNBS2NBaVF6dGFxYXFTVDhOQjBHY1U0MjZSK3ZEYTcra0NnWUVBbFVRMlQzZE1KN1Q2VVgvOHhZRGoKMU9iR0liVDIxYTI1OWk0TGMxamVvMFNxTWM5L2gyR3lmeWZ2VXpaZFdpaEltSmVsN0VMcnRHQ281bkdPUXlmMgo5TjZEZytTL241YjNiWnlzUjZqeWlzQ1hTSnBlUjRwWmZFQjhDVDQ0a2twblNQZ3ZDWXU0VTRabE9LSHdJeFBoClpJL1hCNFkxOU5DWFMrZ0VMcTZZWmIwQ2dZRUExVVpsNEVlN2tOMXUyekNzVVQ5K3lPK2pWaTBDQXNOU1h5ZWgKVHFtK0Z3UXorbExuV1g4OE5QaTJhUHVkU2RYcmZ5R2JmamZFNks4OEFuMWxBOUUwVDBRYWkyc3JlcUxKSmIvVQpuQk1Vb0tDbWU2bDdpNEpsWUJBeU9kdU44ZnZHejc4U2Q0NGxLSTljTXZaRWQ5WHNBcnBqdFZwcXNza3ZQa0lmClVMZGNTOGtDZ1lCejZ2QlJqc2VlZnpySXVSRjR5UWorcUhZM2cxYXR2VDBMYlh5MWVvTk5RdW92ZXhoZmduTHoKangyVVB0Z29VMjlrMUpOWWhzMjByWVBhS1NDa2c5ZjNqUk1NNmZOc2M4QWQ5RFNJa3FaeS9rQkI0YjJPMDdKdwo2cVBRZEhyemJDK0Q2VUZhakFJMEFoVVV1WUZXbFRPaW5uLzlEc0p4Ky9CbEZSQzJFZTJtNEE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
转码
echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRY3d4c0NtVmNPV2RpZWJTSVREcU9kakFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05Nakl4TWpJNE1EWXdOVE13V2hjTk1qTXhNakk0TURZdwpOVE13V2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRQ3g0YVBFbkJ5K2xicElsSytyZmlsdndIWkJRMzM1a3lxMThKWHhpcDl1SFQ0VkExUzEKbG45anVtQUR4Smk4QnhWZStmVlBDWW5UeGJoNWJua3pqdm9XekptOTlMRUY5UDJPdW5VZnZCWGt6VFZEK0o1TgpBZjhyMmtqMFRTck5yZVVKL3JwR01mT1V1MFRSSjg2N2g1WHg1aEpBeDNlWlk4LzRFWmZQOVFGRlFsVGRxRDFBCmlkUkRydjFCSGJIajNxVjU3ckI4L3VxMVJ1a2hjQUZodlZXKzMvdmNkelBaS0hjc0c4d3VhNml2ekExK096a1gKTFA2K2wvcnlPektWWG9kU3ZHV1RmVzJpSlFEUjJqSWliR1loSUF3SDIrZlU4MVJrdGNsS050b2hZYW5aYjdOdQpTcTlzOFB1b3Y2V1JDMWpwRUd6cjhtR2hKTEhtcHJMNlRXeDdBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVZc2xqdmpDZ3F5R2NRZ1ZWS0s5TnIreU8yWll3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFKd1NkMkFuMmpEdHp6NUIvOEl2UzliamxVN3hLVlR0M1pWWkc0N0R5aFJXTmRpVEs1NFF3dFdhCmtCYUZrV2xTQWJaUk5iOWwyak85NEFMelRCb0ttVU04QTBlZFBQbXdvTFFaZlV2NVp6SCtGM05PenNVT1I5dFIKbnhva3lkSU14WUlpQWp1YThudXRLK0g2TklsRDRiVVA1MVZSNFJsNjBhdHhSWi81OW1HMWw1ZUNVSWtObVJyaQpwY2FQTjBBa3NzR2dCNmpxK3VXMXhPU0l1M21idVl3amdzTEpqWTFkNkNuS1IvQ1RRZnRBKzB2NFNVRTJBTFRGClg0TEFRVXFsM0F2NU8wWkxFQXAzNE5aR0xGNCtnRmJuYi84MmdYTlBuY2IzQkxOR0tKVE5zL0RGTTBkYlgwajcKY3VIMHQvT01oV2YyZ084eTNBMDB6dkIzcTVZSkIwdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=' | base64 -d > ca.crt-----BEGIN CERTIFICATE-----
MIIDEzCCAfugAwIBAgIQcwxsCmVcOWdiebSITDqOdjANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwloYXJib3ItY2EwHhcNMjIxMjI4MDYwNTMwWhcNMjMxMjI4MDYw
NTMwWjAUMRIwEAYDVQQDEwloYXJib3ItY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQCx4aPEnBy+lbpIlK+rfilvwHZBQ335kyq18JXxip9uHT4VA1S1
ln9jumADxJi8BxVe+fVPCYnTxbh5bnkzjvoWzJm99LEF9P2OunUfvBXkzTVD+J5N
Af8r2kj0TSrNreUJ/rpGMfOUu0TRJ867h5Xx5hJAx3eZY8/4EZfP9QFFQlTdqD1A
idRDrv1BHbHj3qV57rB8/uq1RukhcAFhvVW+3/vcdzPZKHcsG8wua6ivzA1+OzkX
LP6+l/ryOzKVXodSvGWTfW2iJQDR2jIibGYhIAwH2+fU81RktclKNtohYanZb7Nu
Sq9s8Puov6WRC1jpEGzr8mGhJLHmprL6TWx7AgMBAAGjYTBfMA4GA1UdDwEB/wQE
AwICpDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDwYDVR0TAQH/BAUw
AwEB/zAdBgNVHQ4EFgQUYsljvjCgqyGcQgVVKK9Nr+yO2ZYwDQYJKoZIhvcNAQEL
BQADggEBAJwSd2An2jDtzz5B/8IvS9bjlU7xKVTt3ZVZG47DyhRWNdiTK54QwtWa
kBaFkWlSAbZRNb9l2jO94ALzTBoKmUM8A0edPPmwoLQZfUv5ZzH+F3NOzsUOR9tR
nxokydIMxYIiAjua8nutK+H6NIlD4bUP51VR4Rl60atxRZ/59mG1l5eCUIkNmRri
pcaPN0AkssGgB6jq+uW1xOSIu3mbuYwjgsLJjY1d6CnKR/CTQftA+0v4SUE2ALTF
X4LAQUql3Av5O0ZLEAp34NZGLF4+gFbnb/82gXNPncb3BLNGKJTNs/DFM0dbX0j7
cuH0t/OMhWf2gO8y3A00zvB3q5YJB0w=
-----END CERTIFICATE-----
将证书保存在docker目录下，创建根域名相同的子目录，这一步建议所有机器都执行
mkdir -p /etc/docker/certs.d/core.harbor.domain
cp ca.crt /etc/docker/certs.d/core.harbor.domain/
6.配置host
建议所有机器执行
将harbor域名解析到ingress-nginx的svc的clusterIP，实现内网使用harbor的功能
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller-admission-nginx ClusterIP 10.106.125.71
ingress-nginx-controller-nginx NodePort 10.108.233.252
echo '10.108.233.252 core.harbor.domain' >> /etc/hosts
三.测试
1.浏览器访问harbor
修改本机host
47.92.*.* core.harbor.domain
查看ingress-nginx暴露的https协议端口，这里我们能看到https协议的端口是32292
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller-admission-nginx ClusterIP 10.106.125.71
ingress-nginx-controller-nginx NodePort 10.108.233.252
浏览器访问
2.服务器内部访问harbor
docker login https://core.harbor.domain
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded