K8s部署eureka注册中心集群

K8s部署SpringCloud eureka集群

一.概述

对于一般的后端微服务来说，在k8s中同时起多个相同的服务来做负载均衡，只需要简单的修改deployment的replicas，增加pod数量，然后通过对外暴露一个service来代理这些pod。

而对于eureka来说，要实现eureka的高可用，那就不是修改replicas这么方便了。由于部署的多个eureka之间需要将自己注册到彼此，因此要做一些特殊改动。

主要是用到了StatefulSet和headless service这两个k8s对象。

StatefulSet

StatefulSet是为了解决有状态服务的问题（对应Deployments和ReplicaSets是为无状态服务而设计），在部署或者扩展的时候要依据定义的顺序依次依次进行（即从0到N-1，在下一个Pod运行之前所有之前的Pod必须都是Running和Ready状态）

Headless Service

Headless Service 和普通service的一个显著的区别是，Headless Service的对应的每一个Endpoints，即每一个Pod，都会有对应的DNS域名。例如：我们可以用过这种域名来访问某个具体的pod：

statefulSetName-0.serviceName.namespace.svc.cluster.local

也就是说我们可以完成通过容器的主机名来访问容器。

这点在eureka的配置文件中defaultZone非常重要。

二.部署

这里eureka集群副本数为3。

eureka服务的yml配置文件

server:
  port: ${PORT:8679}
spring:
  application:
    name: amp-eureka

eureka:
  server:
    enable-self-preservation: true
  instance:
    leaseRenewalIntervalInSeconds: 5
    leaseExpirationDurationInSeconds: 10
    instance-id: ${EUREKA_INSTANCE_HOSTNAME:${spring.application.name}}:${server.port}@${random.long(1000000,9999999)}
    hostname: ${EUREKA_INSTANCE_HOSTNAME:${spring.application.name}}
  client:
#    register-with-eureka: true
#    fetch-registry: true
    registry-fetch-interval-seconds: 5
    serviceUrl:
      defaultZone: ${EUREKA_SERVER:http://127.0.0.1:${server.port}/eureka/}
logging:
  path: logs/
  level:

这里定义的$PORT,${EUREKA_SERVER}都会在eureka的StatefulSet的yaml文件中赋值传入的容器中。

将eureka的镜像打包传入到Harbor服务器192.168.0.88:5000仓库中，待部署时拉取。

eureka的StatefulSet的yaml文件

apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: amp
  name: amp-eureka
  labels:
    app: amp-eureka
spec:
  serviceName: 'amp-eureka-inside'
  replicas: 3
  selector: 
    matchLabels:
      app: amp-eureka
  template:
    metadata:
      labels:
        app: amp-eureka
    spec:
      volumes:
      - name: logs
        nfs:
          path: /home/docker/k8s_yaml/nfs_logs/amp-eureka
          server: 192.168.0.88
      #nodeName: server88
      imagePullSecrets:
        - name: mysecret
      containers:
      - name: amp-eureka
        image: 192.168.0.88:5000/myamp/amp-eureka
     #   imagePullPolicy: Always
        ports:
        - containerPort: 20000
        volumeMounts:
          - name: logs
            mountPath: /opt/app/logs
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: JAVA_OPTS
            value: -XX:+UnlockExperimentalVMOptions
              -XX:+UseCGroupMemoryLimitForHeap
              -XX:MaxRAMFraction=2
              -XX:CICompilerCount=8
              -XX:ActiveProcessorCount=8
              -XX:+UseG1GC
              -XX:+AggressiveOpts
              -XX:+UseFastAccessorMethods
              -XX:+UseStringDeduplication
              -XX:+UseCompressedOops
              -XX:+OptimizeStringConcat
          - name: EUREKA_SERVER
            value: "http://amp-eureka-0.amp-eureka-inside.amp:20000/eureka/,http://amp-eureka-1.amp-eureka-inside.amp:20000/eureka/,http://amp-eureka-2.amp-eureka-inside.amp:20000/eureka/"
          - name: EUREKA_INSTANCE_HOSTNAME
            value: ${POD_NAME}.amp-eureka-inside.amp
          - name: PORT
            value: '20000'
          - name: localdocker_eureka_com
            value: amp-eureka1
          - name: localdocker_git_com
            value: 192.168.0.89
          - name: localdocker_redis_com
            value: 192.168.0.88
          - name: localdocker_db_com
            value: 192.168.0.88
          - name: localdocker_reids_com
            value: 192.168.0.88
          - name: localdocker_zk_com
            value: 192.168.0.88
          - name: localdocker_rabbitmq_com
            value: 192.168.0.88

这里还需要Headless Service资源，供3个eureka集群相互通讯（通过容器的主机名），这也是为什么defaultZone配置会写成是“http://amp-eureka-0.amp-eureka-inside.amp:20000/eureka/,http://amp-eureka-1.amp-eureka-inside.amp:20000/eureka/,http://amp-eureka-2.amp-eureka-inside.amp:20000/eureka/”的原因

Headless Service配置文件

注意，这里的clusterIP为None,这是区别Headless Service与普通Service的地方。

apiVersion: v1
kind: Service
metadata:
  namespace: amp
  name: amp-eureka-inside
  labels:
    app: amp-eureka
spec:
  clusterIP: None
  ports:
    - port: 20000
  selector:
    app: amp-eureka

执行命令启动eureka的集群容器

[docker@server88 ~]$ kubectel create -f amp-eureka-svc-inside.yaml
[docker@server88 ~]$ kubectl create -f cat amp-eureka-statefulset.yaml

查看pod的启动情况,可以发现已经创建了3个POD资源,而且是以amp-eureka为固定字符串，0,1,3…N增长。

此时我们还需要创建两个server资源，分别作用是第一个映射到宿主机端口供外部访问查看，第二个供容器间config，user等等这些服务组件注册访问。

[docker@server88 ~]$ cat  amp-eureka-svc-outside.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: amp
  name: amp-eureka-outside
spec:
  type: NodePort
  ports:
    - port: 20000
      nodePort: 20000
  selector:
    app: amp-eureka
    


[docker@server88 ~]$ cat amp-eureka-svc-other.yaml 
apiVersion: v1
kind: Service
metadata:
  name: amp-eureka
  namespace: amp
spec:
  ports:
    - port: 20000
      targetPort: 20000
  selector:
    app: amp-eureka

通过访问映射http://192.168.0.89:20000/查看集群状况

集群已经相互注册成功。

此时我们还需要在K8s上部署其他服务模块，进一步检测eureka集群是否可用

部署config服务模块

[docker@server88 ~]$ vim  amp-config-deploy.yaml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: amp
  name: amp-config
  labels:
    app: amp-config
spec:
  replicas: 1
  selector:
    matchLabels:
      app: amp-config
  template:
    metadata:
      labels:
        app: amp-config
    spec:
      nodeName: server88
      volumes:
      - name: logs
        nfs:
          path: /home/docker/k8s_yaml/nfs_logs/amp-config
          server: 192.168.0.88
#      nodeName: server101
      imagePullSecrets:
        - name: mysecret
      containers:
      - name: amp-config
        image: 192.168.0.88:5000/myamp/amp-config
        imagePullPolicy: Always
        ports:
        - containerPort: 20140
        volumeMounts:
          - name: logs
            mountPath: /opt/app/logs
        env:
          - name: localdocker_eureka_com
            value: amp-eureka
          - name: localdocker_git_com
            value: 192.168.0.89
          - name: localdocker_redis_com
            value: 192.168.0.88
          - name: localdocker_db_com
            value: 192.168.0.88
          - name: localdocker_reids_com
            value: 192.168.0.88
          - name: localdocker_zk_com
            value: 192.168.0.88
          - name: localdocker_rabbitmq_com
            value: 192.168.0.88

部署user模块

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: amp
  name: amp-user-server
  labels:
    app: amp-user-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: amp-user-server
  template:
    metadata:
      labels:
        app: amp-user-server
    spec:
#      #nodeName: server88
      volumes:
      - name: logs
        nfs:
          path: /home/docker/k8s_yaml/nfs_logs/amp-user-server
          server: 192.168.0.88

    #  hostAliases: 
    #  - ip: 192.168.0.101
    #    hostnames:
    #    - "server101"
      imagePullSecrets:
        - name: mysecret
      containers:
      - name: amp-user-server
        image: 192.168.0.88:5000/myamp/amp-user-server
        imagePullPolicy: Always
        ports:
        - containerPort: 20120
#        resources:
 #         limits: 
  #          cpu: "400m"
   #         memory: "512Mi"
        volumeMounts:
          - name: logs
            mountPath: /opt/app/logs

        env:
          - name: localdocker_eureka_com
            value: amp-eureka
          - name: localdocker_git_com
            value: 192.168.0.89
          - name: localdocker_redis_com
            value: 192.168.0.88
          - name: localdocker_db_com
            value: 192.168.0.88
          - name: localdocker_reids_com
            value: 192.168.0.88
          - name: localdocker_zk_com
            value: 192.168.0.88
          - name: localdocker_rabbitmq_com
            value: 192.168.0.88

所有的模块yaml文件都存在了server88服务器的/home/docker/test下，这里不一一列举

启动所有服务模块

[docker@server88 ~]$ kubectl create -f .

启动完毕后查看资源启动情况

[docker@server88 ~]$ kubectl get all -o wide -n amp

所有服务都已经启动成功

稍等一会，打开erureka的外部页面查看各服务模块的注册情况

所有服务模块都已注册成功~