k8s 搭建redis集群
环境
master01 172.16.248.191
node01 172.16.248.171
node02 172.16.248.173
目录
1.创建NFS存储
2、创建PV
3、创建Configmap
4、创建Headless service
5、创建Redis 集群节点
6、初始化集群
7、创建Service
1.创建NFS存储
（node02）安装NFS
yum -y install nfs-utils rpcbind新建文件/etc/exports
/usr/local/k8s/redis/pv1 *(rw,all_squash)
/usr/local/k8s/redis/pv2 *(rw,all_squash)
/usr/local/k8s/redis/pv3 *(rw,all_squash)
/usr/local/k8s/redis/pv4 *(rw,all_squash)
/usr/local/k8s/redis/pv5 *(rw,all_squash)
/usr/local/k8s/redis/pv6 *(rw,all_squash)创建相应目录
mkdir -p /usr/local/k8s/redis/pv{1..6}给目录赋权限
chmod 777 /usr/local/k8s/redis/pv*启动NFS和rpcbind服务
systemctl start rpcbind
systemctl start nfs安装nfs服务不然启动pod会报错
yum -y install nfs-utils rpcbind2、创建PV
master01
创建pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv1
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv1"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv2
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv2"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv3
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv3"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv4
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv4"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv5
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv5"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv6
spec:
capacity:
storage: 200M
accessModes:
- ReadWriteMany
nfs:
server: 172.16.248.173
path: "/usr/local/k8s/redis/pv6"kubectl create -f pv.yml
3、创建Configmap
新建文件redis.conf
appendonly yes
cluster-enabled yes
cluster-config-file /var/lib/redis/nodes.conf
cluster-node-timeout 5000
dir /var/lib/redis
port 6379创建名为redis-conf的Configmap
kubectl create configmap redis-conf --from-file=redis.conf4、创建Headless service
创建文件headless-service.yml
apiVersion: v1
kind: Service
metadata:
name: redis-service
labels:
app: redis
spec:
ports:
- name: redis-port
port: 6379
clusterIP: None
selector:
app: redis
appCluster: redis-clusterkubectl create -f headless-service.yml
5、创建Redis 集群节点
创建redis.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: redis-app
spec:
serviceName: "redis-service"
replicas: 6
template:
metadata:
labels:
app: redis
appCluster: redis-cluster
spec:
terminationGracePeriodSeconds: 20
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- redis
topologyKey: kubernetes.io/hostname
containers:
- name: redis
image: redis
command:
- "redis-server"
args:
- "/etc/redis/redis.conf"
- "--protected-mode"
- "no"
resources:
requests:
cpu: "100m"
memory: "100Mi"
ports:
- name: redis
containerPort: 6379
protocol: "TCP"
- name: cluster
containerPort: 16379
protocol: "TCP"
volumeMounts:
- name: "redis-conf"
mountPath: "/etc/redis"
- name: "redis-data"
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf"
configMap:
name: "redis-conf"
items:
- key: "redis.conf"
path: "redis.conf"
volumeClaimTemplates:
- metadata:
name: redis-data
spec:
accessModes: [ "ReadWriteMany" ]
resources:
requests:
storage: 200Mkubectl create -f redis.yml
6、初始化集群
开始初始化集群，首先在k8s上创建一个ubuntu的pod，用来作为管理节点
kubectl run -it ubuntu --image=ubuntu --restart=Never /bin/bash进入pod内部先安装一些工具，包括wget,dnsutils，然后下载和安装redis：(以下redis-cluster-manager中操作)
apt-get update
apt-get install -y vim wget python2.7 python-pip redis-tools dnsutils
pip install redis-trib==0.5.1创建只有Master节点的集群:
redis-trib.py create \
`dig +short redis-app-0.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-app-1.redis-service.default.svc.cluster.local`:6379 \
`dig +short redis-app-2.redis-service.default.svc.cluster.local`:6379为每个Master添加Slave:
redis-trib.py replicate \
--master-addr `dig +short redis-app-0.redis-service.default.svc.cluster.local`:6379 \
--slave-addr `dig +short redis-app-3.redis-service.default.svc.cluster.local`:6379
redis-trib.py replicate \
--master-addr `dig +short redis-app-1.redis-service.default.svc.cluster.local`:6379 \
--slave-addr `dig +short redis-app-4.redis-service.default.svc.cluster.local`:6379
redis-trib.py replicate \
--master-addr `dig +short redis-app-2.redis-service.default.svc.cluster.local`:6379 \
--slave-addr `dig +short redis-app-5.redis-service.default.svc.cluster.local`:6379
另外，还可以在NFS上查看Redis挂载的数据
[root@ftp pv3]# ll /usr/local/k8s/redis/pv3
total 12
-rw-r--r-- 1 root root 92 Jun 4 11:36 appendonly.aof
-rw-r--r-- 1 root root 175 Jun 4 11:36 dump.rdb
-rw-r--r-- 1 root root 794 Jun 4 11:49 nodes.conf
7、创建Service
现在进入redis集群中的任意一个节点都可以直接进行操作了，但是为了能够对集群其他的服务提供访问，还需要建立一个service来实现服务发现和负载均衡（注意这里的service和我们之前创建的headless service不是一个东西）
redis-access-service.yaml 如下
apiVersion: v1
kind: Service
metadata:
name: redis-access-service
labels:
app: redis
spec:
ports:
- name: redis-port
protocol: "TCP"
port: 6379
targetPort: 6379
selector:
app: redis
appCluster: redis-cluster
type: NodePort kubectl create -f redis-access-service.yaml
kubectl get svc redis-access-service -o wideNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
redis-access-service NodePort 10.111.137.151 6379:32668/TCP 52m app=redis,appCluster=redis-cluster 外部可通过IP:32668来访问redis