kubernetes(k8s)：构建Helm Chart并部署mychart应用到k8s集群

1. 构建一个 Helm Chart

[kubeadm@server1 helm]$ helm create mychart
Creating mychart
[kubeadm@server1 helm]$ ls
mychart  redis-ha
[kubeadm@server1 helm]$ cd mychart/
[kubeadm@server1 mychart]$ tree .  
.
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

3 directories, 10 files

[kubeadm@server1 mychart]$ ls
charts  Chart.yaml  templates  values.yaml
values.yaml //用于获取变量
chart.yaml //介绍发行版本

[kubeadm@server1 mychart]$ helm lint . // 检测当前目录依赖和模板配置是否正确
==> Linting .
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed

[kubeadm@server2 helm]$ helm package mychart/  // 将应用打包
Successfully packaged chart and saved it to: /home/kubeadm/helm/mychart-0.1.0.tgz
[kubeadm@server2 helm]$ ls
mychart  mychart-0.1.0.tgz  redis-ha

• 编写mychart的应用描述信息

$ vim Chart.yaml 
apiVersion: v2
name: mychart
description: A Helm chart for Kubernetes
type: application
version: 0.1.0
appVersion: latest

• 编写应用部署信息：

$ vim values.yaml
replicaCount: 1

image:
  repository: game2048
  pullPolicy: IfNotPresent
...

[kubeadm@server1 mychart]$ ls
charts  Chart.yaml  templates  values.yaml
[kubeadm@server1 mychart]$ 

主要是编写value.yaml文件

• 修改harbor仓库的配置：

把values.yaml文件指定的镜像提前push到本地的私有镜像仓库中,并修改values.conf文件中的镜像名称，如下格式：

nginx:
  image:
    repository: reg.westos.org	//确保可以从私有仓库下载
    tag: v1.10.1

$ vim values.yaml
expose:
    type: nodePort
    tls:
        enabled: false

harbor仓库默认会动态创建PV持久卷，如果集群中不支持，可以禁用相应的配置：

persistence:
  enabled: false

2. 构建本地chart仓库

方式一：

helm v3 需要外部仓库软件的支持：https://github.com/goharbor/harbor-helm

$ helm repo add harbor https://helm.goharbor.io

$ helm pull harbor/harbor		

方式二：使用我自己搭建的harbor仓库
在harbor私有仓库中新建一个charts项目（公有项目）

可以看出现在仓库还没有chart

将仓库添加到helm：

[kubeadm@server1 helm]$ helm repo add mychart https://reg.westos.org/chartrepo/charts
Error: looks like "https://reg.westos.org/chartrepo/charts" is not a valid chart repository or cannot be reached: Get https://reg.westos.org/chartrepo/charts/index.yaml: x509: certificate signed by unknown authority

//问题：需要证书x509: certificate signed by unknown authority
[kubeadm@server1 helm]$

可以看出报错是缺少证书，可以将证书复制到redhat的全局证书地址

[root@server1 reg.westos.org]# pwd
/etc/docker/certs.d/reg.westos.org
[root@server1 reg.westos.org]# ls  // 当前的证书只有docker能使用
ca.crt

[root@server1 reg.westos.org]# cp ca.crt /etc/pki/ca-trust/source/anchors/ // 将证书放到系统中的认证目录中
[root@server1 reg.westos.org]# update-ca-trust  // 更新ca信任

再次添加：

[kubeadm@server1 reg.westos.org]$ helm repo add mychart https://reg.westos.org/chartrepo/chart
"mychart" has been added to your repositories
[kubeadm@server1 reg.westos.org]$

添加成功，查看仓库

[kubeadm@server1 helm]$ helm repo list
NAME   	URL                                      
stable 	http://mirror.azure.cn/kubernetes/charts/
mychart	https://reg.westos.org/chartrepo/charts  // 我自己的私有仓库，https://reg.westos.org/chartrepo/charts
//解释https://reg.westos.org/是你的私有仓库地址，chartrepo固定写法，charts是你在私有仓库中创建的项目

3. 安装push插件

将chart push到helm仓库需要push插件，这个插件有两种安装方式

（1）在线安装

 helm plugin install https://github.com/chartmuseum/helm-push	
 //在线安装，注意需要先安装git

（2）在线安装比较慢，也可以使用离线安装的方式

需要helm-push_0.8.1_linux_amd64.tar.gz插件包

[kubeadm@server1 helm]# helm env		//获取插件目录
HELM_BIN="helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBECONTEXT=""
HELM_KUBETOKEN=""
HELM_NAMESPACE="default"
HELM_PLUGINS="/root/.local/share/helm/plugins"		//插件目录
HELM_REGISTRY_CONFIG="/root/.config/helm/registry.json"
HELM_REPOSITORY_CACHE="/root/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/root/.config/helm/repositories.yaml"
 
[kubeadm@server1 helm]# mkdir -p ~/.local/share/helm/plugins/helm-push				//创建插件目录
[kubeadm@server1 ~]# cd ~/.local/share/helm/plugins
[kubeadm@server1 plugins]# ls
helm-push

解压插件的安装包到插件目录

[kubeadm@server1 ~]# tar zxf helm-push_0.8.1_linux_amd64.tar.gz -C ~/.local/share/helm/plugins/helm-push
[kubeadm@server1 ~]# cd ~/.local/share/helm/plugins/helm-push
[kubeadm@server1 helm-push]# ls
bin  LICENSE  plugin.yaml
[kubeadm@server1 helm-push]# helm push --help		//测试插件是否安装成功

现在可以进行push

[kubeadm@server1 helm]$ ls
mychart  mychart-0.1.0.tgz  redis-ha
[kubeadm@server1 helm]$ helm push mychart-0.1.0.tgz mychart -u admin -p westos  
//将mychart包放到我们的mychart仓库中
Pushing mychart-0.1.0.tgz to mychart...
Done.
[kubeadm@server1 helm]$ 

push成功，其中的用户名和密码为harbor仓库的用户和密码。

现在在harbor仓库可以看到上传的chart
在本地还需要更新才可以查找到

[kubeadm@server1 helm]$ helm repo update    //更新repo
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "mychart" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈ 
[kubeadm@server1 helm]$

[kubeadm@server1 helm]$ helm search repo mychart   //可以查看到mychart中的打包文件
NAME           	CHART VERSION	APP VERSION	DESCRIPTION                
mychart/mychart	0.1.0        	1.16.0     	A Helm chart for Kubernetes
[kubeadm@server1 helm]$

4 .部署mychart应用到k8s集群

查看部署参数：

[kubeadm@server1 helm]$ helm show values mychart/mychart 
affinity: {}
autoscaling:
  enabled: false
  maxReplicas: 100
  minReplicas: 1
  targetCPUUtilizationPercentage: 80
fullnameOverride: ""
image:
  pullPolicy: IfNotPresent
  repository: k8s/myapp
  tag: v1
imagePullSecrets: []
ingress:
  annotations: {}
  enabled: false
  hosts:
  - host: chart-example.local
    paths: []
  tls: []
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
replicaCount: 1
resources: {}
securityContext: {}
service:
  port: 80
  type: ClusterIP
serviceAccount:
  annotations: {}
  create: true
  name: ""
tolerations: []

部署：

[kubeadm@server1 helm]$ helm install test mychart/mychart --debug  // 部署mychart应用到k8s集群

其中test为名称，mychart/mychart 为仓库名称，也可以加–dry-run表示做调试，–debug表示输出部署过程。

部署完成后查看

[kubeadm@server1 helm]$ helm list
NAME	NAMESPACE	REVISION	UPDATED                               	STATUS  	CHART        	APP VERSION
test	default  	1       	2020-05-12 23:01:29.93227763 +0800 CST	deployed	mychart-0.1.0	1.16.0    

5. 更新版本(升级)

更新之前查看

[kubeadm@server1 helm]$ kubectl get pod
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-6bf974db79-kd987   1/1     Running   0          87m
test-mychart-5d77476865-7pv6p             1/1     Running   0          3m2s
[kubeadm@server1 helm]$ kubectl get pod -o wide
NAME                                      READY   STATUS    RESTARTS   AGE    IP             NODE      NOMINATED NODE   READINESS GATES
nfs-client-provisioner-6bf974db79-kd987   1/1     Running   0          87m    10.244.2.87    server3   <none>           <none>
test-mychart-5d77476865-7pv6p             1/1     Running   0          3m9s   10.244.1.156   server2   <none>           <none>
[kubeadm@server1 helm]$ curl 10.244.1.156      //可以访问test-mychart，说明部署成功
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[kubeadm@server1 helm]$ curl 10.244.1.156
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[kubeadm@server1 helm]$

可以看出myapp的版本是v1

更改Chart.yaml文件更改版本号为0.2.0

vim mychart/Chart.yaml

更改变量文件values.yaml 更改镜像版本为v2：

vim mychart/values.yaml

打包：

[kubeadm@server1 helm]$ helm package mychart // 打包
Successfully packaged chart and saved it to: /home/kubeadm/helm/mychart-0.2.0.tgz

[kubeadm@server1 helm]$ ls
mychart  mychart-0.1.0.tgz  mychart-0.2.0.tgz  redis-ha

可以看出自动根据版本文件中的信息打包了0.2.0的压缩包，接下来进行push：

[kubeadm@server1 helm]$ helm push mychart-0.2.0.tgz mychart -u admin -p redhat  // 上传新版本
Pushing mychart-0.2.0.tgz to mychart...
Done.

查看仓库
本地查看

[kubeadm@server1 helm]$ helm repo update   //更新仓库
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "mychart" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈ 

[kubeadm@server1 helm]$ helm search repo mychart
NAME           	CHART VERSION	APP VERSION	DESCRIPTION                
mychart/mychart	0.2.0        	1.16.0     	A Helm chart for Kubernetes
[kubeadm@server1 helm]$

可以看出0.2.0版本已经可以查找到了。

也可以加-l选项查看每个版本

[kubeadm@server1 helm]$ helm search repo mychart -l     //两个mychart版本
NAME           	CHART VERSION	APP VERSION	DESCRIPTION                
mychart/mychart	0.2.0        	1.16.0     	A Helm chart for Kubernetes
mychart/mychart	0.1.0        	1.16.0     	A Helm chart for Kubernetes
[kubeadm@server1 helm]$

进行更新

[kubeadm@server1 helm]$ helm upgrade test mychart/mychart   //更新
Release "test" has been upgraded. Happy Helming!
NAME: test
LAST DEPLOYED: Thu May 14 22:34:16 2020
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=mychart,app.kubernetes.io/instance=test" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace default port-forward $POD_NAME 8080:80
[kubeadm@server1 helm]$

查看

[kubeadm@server1 helm]$ helm list      //已经更新到0.2.0版本
NAME	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART        APP VERSION
test	default  	2       	2020-05-14 22:34:16.816615124 +0800 CST	deployed	mychart-0.2.01.16.0     
[kubeadm@server1 helm]$

可以看到chart已经更新到0.2.0

测试查看版本

6. 版本回滚

可以使用以下命令查看部署历史以便回滚

[kubeadm@server1 helm]$ helm history test 
REVISION	UPDATED                 	STATUS    	CHART        	APP VERSION	DESCRIPTION     
1       	Thu May 14 21:08:47 2020	superseded	mychart-0.1.0	1.16.0     	Install complete
2       	Thu May 14 22:34:16 2020	deployed  	mychart-0.2.0	1.16.0     	Upgrade complete
[kubeadm@server1 helm]$ 

使用rollback选项进行回滚

[kubeadm@server1 helm]$ helm rollback test 1    //回滚到version 1
Rollback was a success! Happy Helming!
[kubeadm@server1 helm]$ helm history test 
REVISION	UPDATED                 	STATUS    	CHART        	APP VERSION	DESCRIPTION     
1       	Thu May 14 21:08:47 2020	superseded	mychart-0.1.0	1.16.0     	Install complete
2       	Thu May 14 22:34:16 2020	superseded	mychart-0.2.0	1.16.0     	Upgrade complete
3       	Thu May 14 22:39:41 2020	deployed  	mychart-0.1.0	1.16.0     	Rollback to 1   

[kubeadm@server1 helm]$ helm list  // mychart-0.1.0，标识回滚成功
NAME	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART        	APP VERSION
test	default  	3       	2020-05-12 23:18:28.493052781 +0800 CST	deployed	mychart-0.1.0	1.16.0     

可以看到pod的版本已经回滚到v1.

7. 卸载应用

[kubeadm@server1 helm]$ helm uninstall test  // 卸载test
release "test" uninstalled