k8s 部署syslog-kafka-es

1、部署三节点的Kafka、zookeeper，elasticsearch

2、部署kafka-ui

3、k8s部署springboot程序syslog-kafka-es

 

 kafka数据

 

流程：

一、syslog-kafka-es程序启动，启动514端口监听，接收syslog日志，netty实现数据接收。

二、日志处理、解析，avro格式转换等，存储kafka的demo topic上。

三、开启消费线程，监听kafka的demo topic。消费处理，解析，格式转换，存储到elasticsearch。

apiVersion: v1
kind: Service
metadata:
  name: syslog-kafka-es
  labels:
    service: syslog-kafka-es
spec:
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      protocol: TCP
      nodePort: 30180
    - name: debug
      port: 12306
      targetPort: 12306
      protocol: TCP
      nodePort: 30181
    - name: log
      port: 514
      targetPort: 514
      protocol: UDP
      nodePort: 30182
  type: NodePort
  selector:
    app: syslog-kafka-es
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: syslog-kafka-es
  labels:
    app: syslog-kafka-es
spec:
  replicas: 1
  selector:
    matchLabels:
      app: syslog-kafka-es
  template:
    metadata:
      labels:
        app: syslog-kafka-es
    spec:
      containers:
        - name: syslog-kafka-es
          imagePullPolicy: IfNotPresent
          image: zhxl1989/syslog-kafka-es:latest
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP
            - containerPort: 12306
              name: debug
              protocol: TCP
            - containerPort: 514
              name: log
              protocol: UDP
          resources:
            requests:
              memory: 1Gi
            limits:
              memory: 1Gi
          volumeMounts:
            - mountPath: /etc/localtime
              name: volume-localtime
      volumes:
        - hostPath:
            path: /etc/localtime
            type: ''
          name: volume-localtime

java.net.UnknownHostException: es-0.es-headless.default.svc.cluster.local,es-1.es-headless.default.svc.cluster.local,es-2.es-headless.default.svc.cluster.local: Name or service not known, executionId: 2