019.Kubernetes二进制集群存储longhorn

一 Longhorn存储部署

1.1 Longhorn概述

Longhorn是用于Kubernetes的开源分布式块存储系统。

提示：更多介绍参考：https://github.com/longhorn/longhorn。

1.2 Longhorn部署

  1 [root@master01 ~]# cd /opt/k8s/work/
  2 [root@master01 work]# source /root/environment.sh
  3 [root@master01 work]# for all_ip in ${ALL_IPS[@]}
  4   do
  5     echo ">>> ${all_ip}"
  6     ssh root@${all_ip} "yum -y install iscsi-initiator-utils &"
  7   done

提示：本步骤操作仅需要在master01节点操作。

  1 [root@master01 work]# mkdir longhorn
  2 [root@master01 work]# cd longhorn/
  3 [root@master01 longhorn]# wget \
  4 https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
  5 [root@master01 longhorn]# vi longhorn.yaml

  1 #……
  2 ---
  3 kind: Service
  4 apiVersion: v1
  5 metadata:
  6   labels:
  7     app: longhorn-ui
  8   name: longhorn-frontend
  9   namespace: longhorn-system
 10 spec:
 11   type: NodePort			#修改为nodeport
 12   selector:
 13     app: longhorn-ui
 14   ports:
 15   - port: 80
 16     targetPort: 8000
 17     nodePort: 30002
 18 ---
 19 ……
 20 kind: DaemonSet
 21 ……
 22         imagePullPolicy: IfNotPresent
 23 ……
 24 #……

  1 [root@master01 longhorn]# kubectl apply -f longhorn.yaml
  2 [root@master01 longhorn]# kubectl -n longhorn-system get pods -o wide

提示：若部署异常可删除重建，若出现无法删除namespace，可通过如下操作进行删除：

  1 wget https://github.com/longhorn/longhorn/blob/master/uninstall/uninstall.yaml
  2 rm -rf /var/lib/longhorn/
  3 kubectl apply -f uninstall.yaml
  4 kubectl delete -f longhorn.yaml

1.5 动态sc创建

提示：默认longhorn部署完成已创建一个sc，也可通过如下手动编写yaml创建。

  1 [root@master01 longhorn]# kubectl get sc
  2 NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
  3 ……
  4 longhorn               driver.longhorn.io      Delete          Immediate              true                   15m
  5 [root@master01 longhorn]# vi longhornsc.yaml

  1 kind: StorageClass
  2 apiVersion: storage.k8s.io/v1
  3 metadata:
  4   name: longhornsc
  5 provisioner: rancher.io/longhorn
  6 parameters:
  7   numberOfReplicas: "3"
  8   staleReplicaTimeout: "30"
  9   fromBackup: ""

  1 [root@master01 longhorn]# kubectl create -f longhornsc.yaml

1.6 测试PV及PVC

  1 [root@master01 longhorn]# vi longhornpod.yaml

  1 apiVersion: v1
  2 kind: PersistentVolumeClaim
  3 metadata:
  4   name: longhorn-pvc
  5 spec:
  6   accessModes:
  7     - ReadWriteOnce
  8   storageClassName: longhorn
  9   resources:
 10     requests:
 11       storage: 2Gi
 12 ---
 13 apiVersion: v1
 14 kind: Pod
 15 metadata:
 16   name: longhorn-pod
 17   namespace: default
 18 spec:
 19   containers:
 20   - name: volume-test
 21     image: nginx:stable-alpine
 22     imagePullPolicy: IfNotPresent
 23     volumeMounts:
 24     - name: volv
 25       mountPath: /data
 26     ports:
 27     - containerPort: 80
 28   volumes:
 29   - name: volv
 30     persistentVolumeClaim:
 31       claimName: longhorn-pvc

  1 [root@master01 longhorn]# kubectl apply -f longhornpod.yaml
  2 [root@master01 longhorn]# kubectl get pods -o wide
  3 [root@master01 longhorn]# kubectl get pvc
  4 [root@master01 longhorn]# kubectl get pv

提示：本步骤操作仅需要在master01节点操作。

1.7 Ingress暴露Longhorn

  1 [root@master01 longhorn]# yum -y install httpd-tools
  2 [root@master01 longhorn]# htpasswd -c auth xhy			#创建用户名和密码

提示：也可通过如下命令创建：

  1 [root@master01 longhorn]# kubectl -n longhorn-system create secret generic longhorn-basic-auth --from-file=auth
  2 
  3 [root@master01 longhorn]# vi longhorn-ingress.yaml		#创建ingress规则

  1 apiVersion: networking.k8s.io/v1beta1
  2 kind: Ingress
  3 metadata:
  4   name: longhorn-ingress
  5   namespace: longhorn-system
  6   annotations:
  7     kubernetes.io/ingress.class: "nginx"
  8     nginx.ingress.kubernetes.io/auth-type: basic
  9     nginx.ingress.kubernetes.io/auth-secret: longhorn-basic-auth
 10     nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
 11 spec:
 12   rules:
 13   - host: longhorn.odocker.com
 14     http:
 15       paths:
 16       - path: /
 17         backend:
 18           serviceName: longhorn-frontend
 19           servicePort: 80

  1 [root@master01 longhorn]# kubectl apply -f longhorn-ingress.yaml

提示：本步骤操作仅需要在master01节点操作。

1.8 确认验证

浏览器访问：longhorn.odocker.com，并输入账号和密码。

登录查看。

附加 Helm部署

附0.1 helm安装

Helm 是 Kubernetes 的软件包管理工具，此处作为建议项部署，更多helm参考《053.集群管理-Helm工具》。

  1 [root@master01 ~]# cd /opt/k8s/work/
  2 [root@master01 work]# mkdir /opt/k8s/work/helm
  3 [root@master01 work]# source /root/environment.sh
  4 [root@master01 work]# wget http://down.linuxsb.com:8888/helm-v3.2.3-linux-amd64.tar.gz
  5 [root@master01 work]# tar -zxvf helm-v3.2.3-linux-amd64.tar.gz -C /opt/k8s/work/helm
  6 [root@master01 work]# for master_ip in ${MASTER_IPS[@]}
  7   do
  8     echo ">>> ${master_ip}"
  9     scp -rp /opt/k8s/work/helm/linux-amd64/helm root@${master_ip}:/opt/k8s/bin/
 10     ssh root@${master_ip} "chmod +x /opt/k8s/bin/*"
 11     ssh root@${master_ip} "helm version"
 12     ssh root@${master_ip} "echo 'source <(helm completion bash)' >> $HOME/.bashrc"
 13   done

附0.2 helm安装

  1 [root@master01 work]# helm repo add brigade https://brigadecore.github.io/charts
  2 [root@master01 work]# helm repo add stable https://kubernetes-charts.storage.googleapis.com/				#添加官方repo
  3 [root@master01 work]# helm repo add bitnami https://charts.bitnami.com/bitnami
  4 [root@master01 work]# helm repo list			#查看repo

提示：本步骤操作仅需要在master01节点操作。