k8s经常使用命令记录
k8s搭建
我的本地服务器
INSTALL_K3S_SKIP_DOWNLOAD=true \
K3S_URL=https://10.211.55.4:6443 \
K3S_TOKEN=K1047fd86f774821f115665b59ffa67fd0695b0e7e1f42767c0968362281c83cdfc::server:317e2d1e92bda510675f09b55a8775ea \
./install.sh
查看启动情况
watch kubectl get node
查看配置列表
ps -ef | grep containerd
在每个节点上创建 每个节点上新建/etc/rancher/k3s/registries.yaml文件
写下如下内容
mirrors:
docker.io:
endpoint:
- "https://fsp2sfpr.mirror.aliyuncs.com/"
然后主节点重启
systemctl restart k3s
从节点重启
systemctl restart k3s-agent
每个节点查看镜像是否生效了,底部有镜像信息
cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
查看有哪些镜像
kubectl get pods --all-namespaces -o jsonpath="{..image}" | tr -s '[[:space:]]' '\n' | sort | uniq
尝试装一个pod
kubectl run mynginx --image=nginx:1.22
watch kubectl get pod
查看日志
kubectl logs -f mynginx
查看具体信息
kubectl describe pod mynginx
删除
kubectl delete pod mynginx
查看详细信息
kubectl get node -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mynginx 1/1 Running 0 25s 10.42.0.4 k8s-master
创建副本
kubectl create deployment nginx-deploy --image=nginx:1.22 --replicas=3
查看deploy
kubectl get deploy
删除
kubectl delete deploy nginx-deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 3/3 3 3 2m39s
查看replicaSet
kubectl get replicaSet
[root@k8s-master ~]# kubectl get replicaSet
NAME DESIRED CURRENT READY AGE
nginx-deploy-855866bb46 3 3 3 4m9s
里面的副本集是多少 对应的name ，滚动更新的时候可以看这个name变化
动态扩容和缩容
将副本集调整为2
kubectl scale deploy nginx-deploy --replicas=2
监听
kubectl get deploy --watch
自动扩容和缩容
kubectl autoscale deployment/nginx-deploy --min=1 --max=10 --cpu-percent=75
并设置它的最小 Pod 数量为 3，最大 Pod 数量为 10，CPU 利用率的目标百分比为 75%。在 CPU 利用率超过 80% 时，将增加 Pod 数量以满足负载要求
监听节点变化
kubectl get deploy --watch
查看自动auto扩容和缩容
kubectl get hpa
获取自动扩容和缩容的详细信息
[root@k8s-master ~]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-deploy Deployment/nginx-deploy /75% 3 10 3 116
NAME: HPA资源的名称
REFERENCE: HPA控制的目标资源，通常是Deployment或ReplicaSet名称和kind
TARGETS: 水平自动扩展的资源指标，包括CPU利用率和内存利用率
MINPODS: HPA自动缩放时，最小允许的Pod数量
MAXPODS: HPA自动缩放时，最大允许的Pod数量
REPLICAS: 当前正在运行的Pod数量
AGE: HPA资源创建的时间
删除自动扩容和缩容
kubectl delete hpa nginx-deploy
滚动更新
kubectl get deploy -owide
1.2.2更新成 1.2.3版本
kubectl set image deployment/nginx-deploy nginx=nginx:1.2.3
同时另外一个窗口监听副本name变化
kubectl set image deployment/nginx-deploy nginx=nginx:1.2.3
然后在查看副本信息
kubectl get deploy -owide
回滚操作
查看当前版本
kubectl get deploy -owide
kubectl rollout history deploy/nginx-deploy
输出:
deployment.apps/nginx-deploy
REVISION CHANGE-CAUSE
1
2
查看具体信息
kubectl rollout history deploy/nginx-deploy revision=1
会输出具体信息
回滚具体操作
kubectl rollout undo deploy/nginx-deploy --to-revision=1
service
kubectl expose deploy/nginx-deploy --name=nginx-service --port=9091 --target-port=80
--port 宿主机端口
--target-port 容器端口
查看
kubectl get service
输出
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 443/TCP 26d
ngnginx-service ClusterIP 10.43.129.212 9091/TCP 7s
其中 clusterIP 为容器里面的ip,集群内部可以测试
curl 10.43.129.212:9091 输出nginx
验证service pod里面访问
kubectl run test_nginx -it --image=nginx:1.22 --rm -- bash
创建一个临时的 pod
kubectl get pod
会显示一个临时的pod
kubectl get service
输出
ngnginx-service ClusterIP 10.43.129.212 9091/TCP
curl测试
curl ngnginx-service:9091
返回响应的结果
service详解
kubectl describe service ngnginx-service
endpoints 有三个地址
kubectl get pod
选择其中一个进入
操作命令如下
[root@k8s-master ~]# kubectl exec -it nginx-deploy-855866bb46-lwq4n -- bash
root@nginx-deploy-855866bb46-lwq4n:/# cd /usr/share/nginx/html/
root@nginx-deploy-855866bb46-lwq4n:/usr/share/nginx/html# ls
50x.html index.html
root@nginx-deploy-855866bb46-lwq4n:/usr/share/nginx/html# echo gitxuzan > index.html
root@nginx-deploy-855866bb46-lwq4n:/usr/share/nginx/html#
curl 10.43.129.212:9091
会有三分之1 输出gitxuzan 负载均衡
网络类型有
ClusterIp 默认网络
NodePort 集群外部主机访问到内部
ExternalName: 将集群外部网络引入集群内部
LoadBalancer: 使用云提供商的负载均衡器向外部暴露服务
给service 创建一个 NodePort
kubectl expose deploy/nginx-deploy --name=nginx-outside --type=NodePort --port=9091 --target-port=80
然后查看
kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 > 443/TCP 26d
ngnginx-service ClusterIP 10.43.129.212 > 9091/TCP 45m
nginx-outside NodePort 10.43.26.27 > 9091:32131/TCP 36s
再浏览器访问
宿主机ip:32131 能访问到数据
例如 curl 10.211.55.4:32131
例如 curl 10.211.55.5:32131
例如 curl 10.211.55.6:32131
命名空间 Namespace
用于deploy,service,pod
[root@k8s-master ~]# kubectl get namespace
NAME STATUS AGE
default Active 26d
kube-system Active 26d 控制平面和node
kube-public Active 26d 自动创建公共命名空间,整个集群公用可见可读资源放到这个空间
kube-node-lease Active 26d kubectl get lease -A 检测每个节点是否发生故障
查看命名空间运行的pod
kubectl get pod -A
创建一个命名空间
创建一个命名空间
kubectl create ns mydevelop
在mydevelop下运行一个nginx
kubectl run nginx --image=nginx:1.22 -n=mydevelop
查看pod
[root@k8s-master ~]# kubectl get pod -n=mydevelop
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 3m9s
把创建好的命名空间设置为默认的 mydevelop
ktbectl config set-context $(kubectl config current-context) --namespace=mydevelop
声明式的配置 yaml
| namespaces | 缩写 | Kind |
|---|---|---|
| namespaces | ns | Namespace |
| nodes | no | Node |
| pods | po | Pod |
| services | svc | Service |
| deveployments | deploy | Deveployment |
| replicasets | rs | ReplicaSet |
| statefulsets | sts | StatefulSet |
kubernetes yaml模板
https://kubernetes.io/zh-cn/docs/concepts/workloads/pods/
例如创建简单pod模板 pods/simple-pod.yaml
apiVersion: v1
kind: Pod
metadata: # 元数据
name: nginx-pod # 元数据pod名称
spec:
containers:
- name: nginx
image: nginx:1.22
ports:
- containerPort: 80
执行命令创建
kubectl apply -f pods/simple-pod.yaml
kubectl get pod
执行命令删除
kubectl delete -f pods/simple-pod.yaml
kubectl get pod
yaml 标签使用查看容器
https://kubernetes.io/zh-cn/docs/concepts/overview/working-with-objects/labels/
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod-label
labels:
environment: production
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.22
ports:
- containerPort: 80
展示标签
kubectl get pod --show-labels=true
根据标签查询
kubectl get pod -l "app=nginx,environment=production"
service yaml 模板
https://kubernetes.io/zh-cn/docs/concepts/services-networking/service/
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
app: nginx # 上面的pod 的labels
ports:
- port: 80
targetPort: 80
nodePort: 30007
执行 kubectl apply -f pods/service-pod.yaml
kubectl get service 查看创建的 nginx-service
查看service 详情
kubectl describe svc/nginx-service 等同于 kubectl describe service nginx-service
查看endpoints
IP
kubectl get pod -l "app=nginx" -owide
service 的 endpoints 和 pod的ip 一致，说明通过label绑定起来了
容器运行时的接口CRI
k8s在 v1.2.4版本开始，把dockershim移除，crictl 指令和docker一样
crictl 命令除了没有导入导出，其他基本都支持和docker一样的命令
导入导出需要指令 ctr
演示ctr 导入导出
docker pull alpine:3.15
docker save alpine:3.15 > alpine-3.15.tar
本地传到服务器
scp alpine-3.15.tar [email protected]:~
在宿主机导入 -n k8s.io 是k8s默认空间镜像namespace
ctr -n k8s.io images import alpine-3.15.tar --platform linux/arm64
crictl images 可以查看到了 alpine:3.15的导入
导出在宿主机上
ctr -n k8s.io images export alpine.tar docker.io/library/alpine:3.15 --platform=linux/arm64
金丝雀发布
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-v1
namespace: dev # 依赖上面的namespace
labels:
app: nginx-deployment-v1
spec:
replicas: 3
selector:
matchLabels: #需要和template.metadata.labels一致
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.22
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: dev
spec:
type: NodePort
selector:
app: nginx # 通过labels 绑定pod
ports:
- port: 80
targetPort: 80
nodePort: 30008
然后通过
kubectl get all -n=dev 可以查看详细信息
kubectl get service -n dev
kubectl describe service xxxxx 查看endpoints
发布一个v2 版本，灰度发布
spec:
replicas: 1
selector:
matchLabels: #需要和template.metadata.labels一致
app: nginx
template:
metadata:
labels:
app: nginx
track: v2v2v2v2 # 测试区分用的
spec:
containers:
- name: nginx-v2
image: docker/getting-started # 可以本地导入过来
ports:
- containerPort: 80
通过
- kubectl get all -n=dev 可以查看详细信息
- kubectl get service -n dev
- kubectl describe service xxxxx 查看endpoints
- 最后 curl xxxxxx:30008 查看是否负载均衡了
- 当灰度发布稳定后，升级新版本
新版本副本调整为3
kubectl scale deploy nginx-deployment-v2 --replicas=3 -n=dev
观察变化
kubectl get all -n dev
下线nginx-deployment-v1版本
kubectl scale deploy nginx-deployment-v1 --replicas=0 -n=dev
- 缺点：不能控制用户和网络去灰度区分
- 清除所有，kubectl delete all --all -n=dev
有状态的pod
- hostPath 几个类型
| hostPath ->type | 缩写 |
|---|---|
| DirectoryOrCreate | 目录不存在就创建 |
| Directory | 挂载已存在的目录,不存在挂载会报错 |
| FileOrCreate | 文件不存在则创建 |
| File | 文件类型，不存在挂载会报错 |
| Socket | 挂载UNIX套接字。例如挂载/var/run/docker.sock进程 |
- 写一个mysql-pod.yaml文件
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"
volumeMounts:
# mysql 镜像目录
- mountPath: /var/lib/mysql
name: data-volume
volumes:
- name: data-volume
hostPath:
# 宿主机的目录
path: /home/mysql/data
# 存储的是目录类型
type: DirectoryOrCreate
存储卷相关文档:
https://kubernetes.io/zh-cn/docs/concepts/storage/volumes/
- 执行 kubectl apply -f mysql-pod.yaml
- 执行 kubectl get pod -owide
ConfigMaps
通过例子把my.cnf 通过configmaps 存储到etcd，然后落到容器上
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"
volumeMounts:
# mysql 镜像目录
- mountPath: /var/lib/mysql
name: data-volume
- mountPath: /etc/mysql/conf.d # https://hub.docker.com/_/mysql
name: configmap-volume
readOnly: true # 只读
volumes:
- name: data-volume
hostPath:
# 宿主机的目录
path: /home/mysql/data
# 存储的是目录类型
type: DirectoryOrCreate
# configmap卷
- name: configmap-volume
configMap:
name: configmap-mysql # 匹配下面metatata
---
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap-mysql
data:
# 类文件键文件名my.cnf
my.cnf: |
[mysqld]
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
init-connect='SET NAMES utf8mb4'
[client]
default-character-set=utf8mb4
[mysql]
default-character-set=utf8mb4
然后测试字符集是否生效
kubectl apply -f mysql-pod.yaml
kubectl exec -it mysql-pod -- bash
查看到 cat /etc/mysql/conf.d/my.cnf
登录mysql查看字符集是否生效
mysql -uroot -p
执行
mysql> SHOW VARIABLES LIKE 'character_set_client';
+----------------------+---------+
| Variable_name | Value |
+----------------------+---------+
| character_set_client | utf8mb4 |
+----------------------+---------+
1 row in set (0.00 sec)
mysql> SHOW VARIABLES LIKE 'character_set_client';
+----------------------+---------+
| Variable_name | Value |
+----------------------+---------+
| character_set_client | utf8mb4 |
+----------------------+---------+
说明设置生效了
secret 使用
文档: https://kubernetes.io/zh-cn/docs/tasks/configmap-secret/managing-secret-using-config-file/
文档: https://kubernetes.io/zh-cn/docs/concepts/configuration/secret/#using-secrets-as-environment-variables
apiVersion: v1
kind: Secret
metadata:
name: mysql-password
type: Opaque
data:
password: MTIzNDU2
---
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
# 对应 name: mysql-password
name: mysql-password
# 对应 password: MTIzNDU2
key: password
optional: false
volumeMounts:
# mysql 镜像目录
- mountPath: /var/lib/mysql
name: data-volume
- mountPath: /etc/mysql/conf.d
name: configmap-volume
readOnly: true
volumes:
- name: data-volume
hostPath:
# 宿主机的目录
path: /home/mysql/data
# 存储的是目录类型
type: DirectoryOrCreate
# configmap卷
- name: configmap-volume
configMap:
name: configmap-mysql
---
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap-mysql
data:
# 类文件键
my.cnf: |
[mysqld]
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
init-connect='SET NAMES utf8mb4'
[client]
default-character-set=utf8mb4
[mysql]
default-character-set=utf8mb4
然后执行验证密码
kubectl apply -f mysql-pod-secret.yaml
创建了secret 查看
kubectl describe secret mysql-password
kubectl exec -it mysql-pod -- bash 进入到pod
mysql -uroot -p
卷
- 临时卷， 跟随pod一起删除
- emptyDir 做缓存或者存储日志
- configMap secret ， downwardAPI
kubectl get cm // config-maps
kubectl get pod -owide 查看部署到那台机器，然后查看/var/lib/kubelet/pods/093d358f-ae75-428b-8374-af85466ab63a/volumes/kubernetes.io~configmap/configmap-volume/my.cnf 实际临时券存储的位置，删除后临时券就对应删除了
查看到了config-map 对象，如果删除 kubectl delete pod mysql-pod ,config-map 对象 还是存在
- 持久卷: 删除pod后， 持久券不被删除
- 本地存储 - hostPath 、local
- 网络存储 - NFS
- 分布式存储 - Ceph(cephfs文件存储,rdb快存储)
https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes/#local
https://kubernetes.io/zh-cn/docs/concepts/storage/volumes/
apiVersion: storage.k8s.io/v1
# 存储类
kind: StorageClass
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: Immediate
---
apiVersion: v1
# 持久卷
kind: PersistentVolume
metadata:
name: example-pv
spec:
capacity:
storage: 2Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
local:
path: /mnt/disks/ssd1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-worker1
声明方式持久卷
https://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: local-storage
accessModes:
# 卷可以被一个节点读写方式挂载，并允许多个节点多个pod访问
- ReadWriteOnce
resources:
requests:
storage: 2Gi
kubectl get pvc 查看bound的storageClassName
查看持久券
- 投射券：projected 卷可以将多个券映射到同一个目录
动态创建持久卷
apiVersion: v1
kind: Secret
metadata:
name: mysql-password
type: Opaque
data:
password: MTIzNDU2
---
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:8.0
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
# 对应 name: mysql-password
name: mysql-password
# 对应 password: MTIzNDU2
key: password
optional: false
volumeMounts:
# mysql 镜像目录
- mountPath: /var/lib/mysql
name: data-volume
- mountPath: /etc/mysql/conf.d
name: configmap-volume
readOnly: true
volumes:
- name: data-volume
persistentVolumeClaim:
## kubectl get pvc 的name
#claimName: task-pv-claimy
# 默认存储 WaitForFirstConsumer 模式 删除pvc时候，会删除pc
claimName: local-pvc-1
# hostPath:
# # 宿主机的目录
# path: /home/mysql/data
# # 存储的是目录类型
# type: DirectoryOrCreate
# configmap卷
- name: configmap-volume
configMap:
name: configmap-mysql
---
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap-mysql
data:
# 类文件键
my.cnf: |
[mysqld]
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
init-connect='SET NAMES utf8mb4'
[client]
default-character-set=utf8mb4
[mysql]
default-character-set=utf8mb4
kubectl get sc 查看 VOLUMEBINDINGMODE
kubectl describe pod mysql-pod 查看持久卷
kubectl delete pvc 会自动删除pv