helm-实战
1 概念
本指南讲述使用Helm来管理k8s集群软件包的基础知识。
Helm 有3个重要概念：
1、helm： 一个命令行客户端工具，主要用于 Kubernetes 应用 chart 的创建、 打包、 发布和管理。
2、Chart：应用描述，一系列用于描述 k8s 资源相关文件的集合。
3、Release：基于 Chart 的部署实体，一个 chart 被 Helm 运行后将会生成对应的一个release；release是在 k8s 中创建出真实运行的资源对象。 一个chart通常可以多次安装到同一个集群中。每次安装都会创建一个新release，每个release都有自己的release name
4、Repoistory 本质上是一个web服务器，该服务器保存了一系列的chart阮籍爱你包以供用户下载，并且提供了一个该Repository的chart包的清单文件以供查询。Helm可以同时管理多个不同的Repository
有了这些概念，我们可以这样解释Helm
Helm将charts安装到Kubernetes中，每个安装创建一个新release。要找到新的chart，可以搜索Helm charts存储库repositories
1.1 安装步骤
helm 下载得当前是最新版本，较老版本删除了tailer，V3和V2的架构对比如下：
V2版本的架构中，Tiller在Kubernetes集群中，Helm Client发请求给Tiller需要经过RBAC认证。而V3版本是Helm通过kubeconfig连接kube-apiserver，避免了使用者去配置RBAC权限。
curl -SLO https://get.helm.sh/helm-v3.4.2-linux-amd64.tar.gz //也可以去官网下载（linux请下载amd）：https://github.com/helm/helm/releases
tar -zxvf helm-v3.4.2-linux-amd64.tar.gz
mv linux-arm64/helm /usr/local/bin/helm
helm version //查看是否安装成功
安装成功之后请添加一个库
//添加微软的repo库
helm repo add stable http://mirror.azure.cn/kubernetes/charts
//添加阿里的repo库
helm repo add aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
更新仓库
helm repo update查看仓库列表
helm repo list
删除仓库
helm repo remove aliyunhelm工具常用的几个处理chart的命令
#创建一个新的chart:
$ helm create mychart
Created mychart/
#编辑完chart后，helm可以将其打包到chart压缩包中：
$ helm package mychart
Archived mychart-0.1.-.tgz
#使用helm来帮助查找chart格式或信息的问题
$ helm list mychart
No issues found
$helm search repo //查看那些charts可用
$ helm search repo mysql //查看mysql相关的repo
NAME CHART VERSION APP VERSION DESCRIPTION
aliyun/mysql 0.3.5 Fast, reliable, scalable, and easy to use open-...
stable/mysql 1.6.9 5.7.30 DEPRECATED - Fast, reliable, scalable, and easy...
stable/mysqldump 2.6.2 2.4.1 DEPRECATED! - A Helm chart to help backup MySQL...
stable/prometheus-mysql-exporter 0.7.1 v0.11.0 DEPRECATED A Helm chart for prometheus mysql ex...
aliyun/percona 0.3.0 free, fully compatible, enhanced, open source d...
aliyun/percona-xtradb-cluster 0.0.2 5.7.19 free, fully compatible, enhanced, open source d...
stable/percona 1.2.3 5.7.26 DEPRECATED - free, fully compatible, enhanced, ...
stable/percona-xtradb-cluster 1.0.8 5.7.19 DEPRECATED - free, fully compatible, enhanced, ...
stable/phpmyadmin 4.3.5 5.0.1 DEPRECATED phpMyAdmin is an mysql administratio...
aliyun/gcloud-sqlproxy 0.2.3 Google Cloud SQL Proxy
aliyun/mariadb 2.1.6 10.1.31 Fast, reliable, scalable, and easy to use open-...
stable/gcloud-sqlproxy 0.6.1 1.11 DEPRECATED Google Cloud SQL Proxy
stable/mariadb 7.3.14 10.3.22 DEPRECATED Fast, reliable, scalable, and easy t...
[root@k8s-master ~]# 1.2 创建缺省chart
$helm create vpc-client //创建vpc-client的chart
$ helm install vpcins vpc-client //安装vpc-client，release名字：vpcins,chart：vpc-chart
NAME: vpcins
LAST DEPLOYED: Wed Jan 6 14:44:13 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=vpc-client,app.kubernetes.io/instance=vpcins" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
# helm status vpcins //追踪release的状态
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
vpcins default 1 2021-01-06 14:44:13.378954443 +0800 CST deployed vpc-client-0.1.0 1.16.0
&helm uninstall vpcins
release "vpcins" uninstalled
& helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSIONtemplate-chart目录的文件结构如下：
├── charts #依赖的chart
├── Chart.yaml #Chart本身的版本和配置信息
├── templates #配置模板目录
│ ├── deployment.yaml
│ ├── _helpers.tpl #用于修改kubernetes object配置的模板
│ ├── hpa.yaml
│ └── ingress.yaml
│ ├── NOTES.txt #helm提示信息
│ ├── serviceaccount.yaml
│ ├── service.yaml #kubenetes service
│ └── tests
└── values.yamltemplates目录用于存放模板文件，模板文件通过 {{ .Values.xxxxxx }} 获取values.yaml中定义的值，例如：{{ .Values.replicas }} 、{{ .Values.image }}。
另外，部署自定义chart的命令是 helm install vpcins vpc-client。 helm install 命令的第一个参数称为 Release，名字可以随意取，模板文件使用 {{ .Release.Name }} 可获取Release名称；第二个参数 vpc-client 是自定义chart的目录名。
下图是deployment.yaml的内容
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "vpc-client.fullname" . }}
labels:
{{- include "vpc-client.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "vpc-client.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "vpc-client.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "vpc-client.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}上图为Deployment的yaml配置文件，其中的双大括号括起来的部分是Go template，其中的Values是在values.yaml文件中定义的
#vlaue.yaml 内容
# Default values for vpc-client.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: nginx
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
比如在Deployment.yaml中定义的容器镜像image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"其中的：
.Values.image.repository就是nginx.Values.image.tag就是空
以上两个变量值是在create chart的时候自动生成的默认值，我们将默认值的镜像地址和tag修改为自己的镜像
1.3 修改yaml
没有特殊要求，一般需要修改的地方有image、service、healthCheck、persistentVolume.mountPaths,下图value.yaml文件作为参考
# Default values for mod-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
global: # 设置后覆盖后面默认的镜像仓库
imageRegistry: ""
imagePullSecrets: []
# - myRegistryKeySecretName
statefulset:
enabled: false
## String to partially override fullname template (will maintain the release name)
##
nameOverride: ""
## String to fully override fullname template
##
fullnameOverride: ""
## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .
## You can change type to `Recreate` or can uncomment `rollingUpdate` specification and adjust them to your usage.
deploymentStrategy: {}
# rollingUpdate:
# maxSurge: 25%
# maxUnavailable: 25%
# type: RollingUpdate
# 副本个数
replicaCount: 1
# 容器image及tag
image:
registry: docker.io
repository: bitnami/nginx
tag: latest
pullPolicy: IfNotPresent # IfNotPresent: 有则不拉（减少流量和操作步骤），Always: 不管tag总拉（适合tag不变时更新）
pullSecrets: []
# - private-registry-key
service:
type: ClusterIP # 一般不用修改
ingressPort: 8080
ports:
http: # 多端口暴露时，复制一段
port: 8080 # Service port number for client-a port.
protocol: TCP # Service port protocol for client-a port.
## env set
## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
env: []
# - name: DEMO_GREETING
# value: "Hello from the environment"
# - name: DEMO_FAREWELL
# value: "Such a sweet sorrow"
## command set
startCommand: []
# - "java -Xdebug -Xnoagent -Djava.compiler=NONE"
# - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
# - "-Djava.security.egd=file:/dev/urandom"
# - "-jar /test.jar"
# - "-Duser.timezone=GMT+08"
## Enable configmap and add data in configmap
config:
enabled: false
subPath: ""
mountPath: /conf
data: {}
############################# 示例 ####################################
## 以下示例，挂载文件至 /conf/app.conf
# enabled: true
# mountPath: /conf/app.conf
# subPath: app.conf # 使用subPath时，上面mountPath路径写文件完整绝对路径
# data:
# app.conf: |-
# appname = example-chart
## 以下示例，挂载多个文件至 /conf/ 下
# enabled: true
# mountPath: /conf # 不使用subPath
# data:
# app.conf: |-
# appname = example-chart
# bpp.conf: |-
# bppname
#
## 挂载多个文件至多个不同路径，需要相应修改 templates/deployment-statefulset.yaml
############################# 示例 ####################################
## To use an additional secret, set enable to true and add data
## 用法同上，不另作说明
secret:
enabled: false
mountPath: /etc/secret-volume
subPath: ""
readOnly: true
data: {}
## liveness and readiness
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
healthCheck:
enabled: true
type: tcp # http/tcp
port: http # 健康检查的端口名或端口
httpPath: '/' # http时必须设置
livenessInitialDelaySeconds: 10 # 初始延迟秒数
livenessPeriodSeconds: 10 # 检测周期，默认值10，最小为1
readinessInitialDelaySeconds: 10 # 初始延迟秒数
readinessPeriodSeconds: 10 # 检测周期，默认值10，最小为1
resources: {}
# 容器资源设置
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
## Node labels and tolerations for pod assignment
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
labels: {}
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
annotations: {}
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistentVolume: # 是否存储持久化
enabled: false
## If defined, storageClassName:
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, azure-disk on
## Azure, standard on GKE, AWS & OpenStack)
##
storageClass: "-"
accessMode: ReadWriteOnce
annotations: {}
# helm.sh/resource-policy: keep
size: 1Gi # 大小
existingClaim: {} # 使用已存在的pvc
mountPaths: []
# - name: data-storage
# mountPath: /config
# subPath: config # 多个路径使用同一个pvc使用subPath，用法同上面config中示例说明
# - name: data-storage
# mountPath: /data
# subPath: data
ingress: # 是否使用nginx暴露域名或端口
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
path: /
hosts:
- chart-example.local
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: []
## Prometheus Exporter / Metrics
##
metrics:
enabled: false
image:
registry: docker.io
repository: nginx/nginx-prometheus-exporter
tag: 0.1.0
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
pullSecrets: []
# - myRegistrKeySecretName
## Metrics exporter pod Annotation and Labels
podAnnotations:
# prometheus.io/scrape: "true"
# prometheus.io/port: "9113"
## Metrics exporter resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
## Uncomment and modify this to run a command after starting the core container.
## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
lifecycle: {}
# preStop:
# exec:
# command: ["/bin/bash","/pre-stop.sh"]
# postStart:
# exec:
# command: ["/bin/bash","/post-start.sh"]
## Deployment additional volumes.
deployment:
additionalVolumes: []
## init containers
## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: {}
# - name: fmp-volume-permission
# image: busybox
# imagePullPolicy: IfNotPresent
# command: ['chown','-R', '200', '/extra-data']
# volumeMounts:
# - name: extra-data
# mountPath: /extra-data
## Additional containers to be added to the core pod.
additionalContainers: {}
# - name: my-sidecar
# image: nginx:latest
# - name: lemonldap-ng-controller
# image: lemonldapng/lemonldap-ng-controller:0.2.0
# args:
# - /lemonldap-ng-controller
# - --alsologtostderr
# - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
# env:
# - name: POD_NAME
# valueFrom:
# fieldRef:
# fieldPath: metadata.name
# - name: POD_NAMESPACE
# valueFrom:
# fieldRef:
# fieldPath: metadata.namespace
# volumeMounts:
# - name: copy-portal-skins
# mountPath: /srv/var/lib/lemonldap-ng/portal/skins
1.4 自定义chart
我们创建第一个模板vpc。
步骤1：首先创建一个vpc-client/templates/vpc.yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpc336步骤2： 安装chart
$ helm install vpcins vpc-client
NAME: vpcins
LAST DEPLOYED: Wed Jan 6 15:45:01 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=vpc-client,app.kubernetes.io/instance=vpcins" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT步骤3： 查看加载的模板
￥helm get manifest vpcins
---
# Source: vpc-client/templates/vpc.yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpc336步骤4： 删除新创建的release
helm delete vpc-client1.4 添加一个简单的模板调用
name: 成资源被认为不太好的做法。名称应该是唯一版本。所以我们希望通过插入release名称来生成一个名称字段。
提示： name: 由于 DNS 系统的限制，该字段限制为 63 个字符。因此，release 名称限制为 53 个字符。
简单修改一下vpc.yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpc{{.Release.Name}}模板指令 {{.Release.Name}} 将 release 名称注入模板。传递给模板的值可以认为是 namespace 对象，其中 dot（.）分隔每个 namespace 元素
Release 前面的前一个小圆点表示我们从这个范围的最上面的 namespace 开始（我们将稍微谈一下 scope）。所以我们可以这样理解 .Release.Name："从顶层命名空间开始，找到 Release 对象，然后在里面查找名为 Name 的对象"。
小技巧：
如果你想测试模板渲染，但实际上没有安装任何东西时，可以使用helm install --debug --dry-run ./mychart。这会将chart发送到k8s做渲染。但是不会安装chart，它会将渲染模板返回，以便可以看到输出
[root@k8s-master ~]# helm install --debug --dry-run vpcins ./vpc-client
install.go:172: [debug] Original chart version: ""
install.go:189: [debug] CHART PATH: /root/vpc-client
NAME: vpcins
LAST DEPLOYED: Wed Jan 6 16:40:05 2021
NAMESPACE: default
STATUS: pending-install
REVISION: 1
---
# Source: vpc-client/templates/vpc.yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpcvpcins
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=vpc-client,app.kubernetes.io/instance=vpcins" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT使用 --dry-run 可以更容易地测试代码，但不能确保 Kubernetes 本身会接受生成的模板。最好不要假定你的 chart 只要 --dry-run 成功而被安装。
参考文档：
【1】https://whmzsu.github.io/helm-doc-zh-cn/
【2】https://helm.sh/