k8s部署集群版redis3.2

创建本地目录用于redis持久化

创建6个持久化文件(三主三从)mkdir -p /Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/{pv1,pv2,pv3,pv4,pv5,pv6}

创建持久卷

创建6个持久卷定义vim redis-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv1
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv1"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv2
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv2"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv3
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv3"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv4
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv4"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv5
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv5"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis32-pv6
spec:
  capacity:
    storage: 200M
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/Users/renzhengxin/IdeaProjects/k8s/redis/v32/data/pv6"

执行创建持久卷：kubectl create -f redis-pv.yaml
查看持久卷：kubectl get pv

创建ConfigMap

创建ConfigMap定义vim redis.conf

appendonly yes
cluster-enabled yes
cluster-config-file /var/lib/redis/nodes.conf
cluster-node-timeout 5000
dir /var/lib/redis
port 6379

执行创建cm：kubectl create configmap redis-conf --from-file=redis.conf
查看cm：kubectl describe cm redis-conf

创建Headless服务

创建服务定义vim redis-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis32-svc
  labels:
    app: redis32
spec:
  ports:
  - name: redis-port
    port: 6379
  clusterIP: None
  selector:
    app: redis32
    appCluster: redis32-cluster

执行创建服务：kubectl create -f redis-svc.yaml
查看服务：kubectl get svc

创建StatefulSet

创建StatefulSet定义vim redis-ss.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis32-app
spec:
  serviceName: "redis32-svc"
  replicas: 6
  selector:
    matchLabels:
      app: redis32
      appCluster: redis32-cluster
  template:
    metadata:
      labels:
        app: redis32
        appCluster: redis32-cluster
    spec:
      terminationGracePeriodSeconds: 20
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - redis32
              topologyKey: kubernetes.io/hostname
      containers:
      - name: redis32
        image: amd64/redis:3.2
        command:
          - "redis-server"
        args:
          - "/etc/redis/redis.conf"
          - "--protected-mode"
          - "no"
        resources:
          requests:
            cpu: "100m"
            memory: "100Mi"
        ports:
            - name: redis
              containerPort: 6379
              protocol: "TCP"
            - name: cluster
              containerPort: 16379
              protocol: "TCP"
        volumeMounts:
          - name: "redis-conf"
            mountPath: "/etc/redis"
          - name: "redis-data"
            mountPath: "/var/lib/redis"
      volumes:
      - name: "redis-conf"
        configMap:
          name: "redis-conf"
          items:
            - key: "redis.conf"
              path: "redis.conf"
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      storageClassName: ""
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 200M

执行创建StatefulSet：kubectl create -f redis-ss.yaml
查看StatefulSet：kubectl get statefulSet
查看Pod：kubectl get po
查看PV：kubectl get pv

初始化redis集群

创建ubuntu容器kubectl run -it ubuntu1604 --image=ubuntu:16.04 --restart=Never /bin/bash
apt-get更新apt-get update
安装必须的服务：apt-get install ruby wget dnsutils
gem安装redis-3.2.0：
wget https://rubygems.org/downloads/redis-3.2.0.gem
gem install redis-3.2.0.gem
下载并解压redis-tirb.rb：
wget http://download.redis.io/releases/redis-3.2.0.tar.gz
tar -xvf redis-3.2.0.tar.gz
初始化redis集群：

./redis-3.2.0/src/redis-trib.rb create --replicas 1 `dig +short redis32-app-0.redis32-svc.default.svc.cluster.local`:6379 `dig +short redis32-app-1.redis32-svc.default.svc.cluster.local`:6379 `dig +short redis32-app-2.redis32-svc.default.svc.cluster.local`:6379 `dig +short redis32-app-3.redis32-svc.default.svc.cluster.local`:6379 `dig +short redis32-app-4.redis32-svc.default.svc.cluster.local`:6379 `dig +short redis32-app-5.redis32-svc.default.svc.cluster.local`:6379

验证集群创建成功

进入任意一个redis pod：
kubectl exec -it redis32-app-2 /bin/bash
/usr/local/bin/redis-cli -c
cluster nodes
cluster info

创建redis-cluster-proxy镜像

创建代理镜像用于集群外部访问。
创建镜像链接：k8s部署redis-cluster-proxy
由于镜像已上传，可直接使用已有镜像：nuptaxin/redis-cluster-proxy:v1.0.0

创建proxy配置文件

创建代理的配置文件：vim proxy-cm.yaml

# Redis-Proxy Config
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-proxy
data:
  proxy.conf: |
    # 配置为Redis Cluster Service
    cluster redis32-svc:6379
    bind 0.0.0.0
    # redis-cluster-proxy 对外暴露端口
    port 7777
    # 线程数量
    threads 8
    daemonize no
    enable-cross-slot yes
    log-level error

创建configMap：
kubectl create -f proxy-cm.yaml
查看configMap：
kubectl describe cm redis-proxy

部署proxy

创建deployment：vim proxy-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis-proxy
  template:
    metadata:
      labels:
        app: redis-proxy
    spec:
      imagePullSecrets:
        - name: harbor
      containers:
        - name: redis-proxy
          image: nuptaxin/redis-cluster-proxy:v1.0.0
          imagePullPolicy: IfNotPresent
          command: ["redis-cluster-proxy"]
          args: 
            - -c
            - /data/proxy.conf # 指定启动配置文件
          ports:
            - name: redis-7777
              containerPort: 7777
              protocol: TCP
          volumeMounts:
            - name: redis-proxy-conf
              mountPath: /data/
      volumes:   # 挂载proxy配置文件
        - name: redis-proxy-conf
          configMap:
            name: redis-proxy

执行创建deploy：kubectl create -f proxy-deploy.yaml
查看deployment：kubectl get deploy
查看pod：kubectl get po

创建集群访问入口

创建服务定义vim redis-access-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis32-access-service
  labels:
    name: mysql32-svc
spec:
  type: NodePort
  ports:
  - port: 7777
    protocol: TCP
    targetPort: 7777
    name: http
    # nodePort的端口号必须大于30000
    nodePort: 30079
  selector:
    app: redis-proxy

执行创建服务：kubectl create -f redis-access-service.yaml
查看服务：kubectl get svc

访问集群

如本地已安装redis客户端，使用以下命令：
redis-cli -h localhost -p 30079 -c
查看集群信息与集群节点使用以下指令：
proxy cluster info
proxy cluster nodes

参考文档：
在K8s上部署Redis 集群
【Redis】——引入redis-cluster-proxy使得Redis Cluster对Kubernetes外部可提供服务
redis-Cluster第三篇redis-trib.rb 搭建&原理篇