K8S安装部署Nacos集群

1 在k8s中部署的难点

在k8s中部署nacos集群和在裸机器上直接部署nacos机器其实差别不大。
最主要的区别是k8s中部署的服务没有固定的ip地址，而nacos集群部署需要配置所有实例的ip

2 解决

• 在k8s中通过StatefulSet和Headless Service为每个nacos实例生成一个唯一的dns地址，
• 创建一个普通Service给可客户端使用

具体实现如下。

3 实现方式

3.1 创建数据库配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
data:
  mysql.host: "192.168.1.3"
  mysql.db.name: "nacos_devtest"
  mysql.port: "3306"
  mysql.user: "nacos"
  mysql.password: "nacos"

3.2 部署Headless Service

Headless Service为每个pod（nacos实例）生成一个DNS地址，用作NACOS_SERVERS配置

apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  labels:
    app: nacos
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  ports:
    - port: 8848
      name: server
      targetPort: 8848
  clusterIP: None
  selector:
    app: nacos

3.3 通过StatefulSet部署nacos

StatefulSet部署方式为每个POD生成固定的名称，如nacos-0、nacos-1、nacos-2等。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos-headless
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: k8snacos
          imagePullPolicy: Always
          image: nacos/nacos-server:latest
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: MODE
              value: "cluster"
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"
  selector:
    matchLabels:
      app: nacos

3.4 部署普通Service（Nginx使用）

apiVersion: v1
kind: Service
metadata:
  name: nacos-service
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/affinity: "true"
    nginx.ingress.kubernetes.io/session-cookie-name: backend
    nginx.ingress.kubernetes.io/load-balancer-method: drr


spec:
  selector:
    app: nacos
  ports:
    - name: web
      port: 80
      targetPort: 8848

3.5 配置Ingress（可选）

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nacos-web
  namespace: default

spec:
  rules:
    - host: nacos-web.nacos-demo.com
      http:
        paths:
          - path: /
            backend:
              serviceName: nacos-service
              servicePort: web