k8s 部署canal admin及server单机服务

目录

1. 前言

2. 部署canal-admin

2.1 数据库初始化

2.2 canal-admin k8s yaml部署文件

3. 部署canal-server单机


1. 前言

  • canal官方文档:https://github.com/alibaba/canal/wiki

主要用途是基于 MySQL 数据库增量日志解析,提供增量数据订阅和消费。

部署清单

软件 版本 备注
canal-admin v1.1.6 镜像:canal/canal-admin:v1.1.6
canal-server v1.1.6 镜像:canal/canal-server:v1.1.6,单机部署
mysql 5.7.19

部署步骤
a. 初始化数据库及创建用户
b. 编写 K8s yaml文件

2. 部署canal-admin


2.1 数据库初始化

 
mysql执行以下sql脚本,创建库 'canal_manager' 用来存放canal-admin的数据,server、instance配置等数据

# 初始化sql脚本
https://raw.githubusercontent.com/alibaba/canal/canal-1.1.6/admin/admin-web/src/main/resources/canal_manager.sql 

# 具体如下
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `canal_manager` /*!40100 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin */;

USE `canal_manager`;

SET NAMES utf8;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for canal_adapter_config
-- ----------------------------
DROP TABLE IF EXISTS `canal_adapter_config`;
CREATE TABLE `canal_adapter_config` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `category` varchar(45) NOT NULL,
  `name` varchar(45) NOT NULL,
  `status` varchar(45) DEFAULT NULL,
  `content` text NOT NULL,
  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for canal_cluster
-- ----------------------------
DROP TABLE IF EXISTS `canal_cluster`;
CREATE TABLE `canal_cluster` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `name` varchar(63) NOT NULL,
  `zk_hosts` varchar(255) NOT NULL,
  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for canal_config
-- ----------------------------
DROP TABLE IF EXISTS `canal_config`;
CREATE TABLE `canal_config` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `cluster_id` bigint(20) DEFAULT NULL,
  `server_id` bigint(20) DEFAULT NULL,
  `name` varchar(45) NOT NULL,
  `status` varchar(45) DEFAULT NULL,
  `content` text NOT NULL,
  `content_md5` varchar(128) NOT NULL,
  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  UNIQUE KEY `sid_UNIQUE` (`server_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for canal_instance_config
-- ----------------------------
DROP TABLE IF EXISTS `canal_instance_config`;
CREATE TABLE `canal_instance_config` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `cluster_id` bigint(20) DEFAULT NULL,
  `server_id` bigint(20) DEFAULT NULL,
  `name` varchar(45) NOT NULL,
  `status` varchar(45) DEFAULT NULL,
  `content` text NOT NULL,
  `content_md5` varchar(128) DEFAULT NULL,
  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  UNIQUE KEY `name_UNIQUE` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for canal_node_server
-- ----------------------------
DROP TABLE IF EXISTS `canal_node_server`;
CREATE TABLE `canal_node_server` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `cluster_id` bigint(20) DEFAULT NULL,
  `name` varchar(63) NOT NULL,
  `ip` varchar(63) NOT NULL,
  `admin_port` int(11) DEFAULT NULL,
  `tcp_port` int(11) DEFAULT NULL,
  `metric_port` int(11) DEFAULT NULL,
  `status` varchar(45) DEFAULT NULL,
  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

-- ----------------------------
-- Table structure for canal_user
-- ----------------------------
DROP TABLE IF EXISTS `canal_user`;
CREATE TABLE `canal_user` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `username` varchar(31) NOT NULL,
  `password` varchar(128) NOT NULL,
  `name` varchar(31) NOT NULL,
  `roles` varchar(31) NOT NULL,
  `introduction` varchar(255) DEFAULT NULL,
  `avatar` varchar(255) DEFAULT NULL,
  `creation_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

SET FOREIGN_KEY_CHECKS = 1;

-- ----------------------------
-- Records of canal_user
-- ----------------------------
BEGIN;
INSERT INTO `canal_user` VALUES (1, 'admin', '6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9', 'Canal Manager', 'admin', NULL, NULL, '2019-07-14 00:05:28');
COMMIT;

SET FOREIGN_KEY_CHECKS = 1;

2.2 canal-admin k8s yaml部署文件


创建部署 deployment.yaml,资源清单包含:service,deployment,configmap,ingress。使用官方canal-admin镜像部署,可以通过环境变量的方式传递数据库配置(而不用修改canal-admin/conf/application.yml文件),因此,configmap存放了数据库等相关配置

canal-admin服务监听端口为:8089


canal-admin 部署文件

apiVersion: v1
kind: Service
metadata:
  name: canal-admin
spec:
  selector:
    app: canal-admin
  ports:
    - name: http
      protocol: TCP
      port: 8089
      targetPort: 8089 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: canal-admin
  labels:
    app: canal-admin
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - canal-admin
        topologyKey: "kubernetes.io/hostname"
  replicas: 1
  selector:
    matchLabels:
      app: canal-admin
  template:
    metadata:
      labels:
        app: canal-admin
    spec:
      restartPolicy: "Always"
      containers:
      - name: canal-admin
        image: canal/canal-admin:v1.1.6
        imagePullPolicy: "IfNotPresent"
        resources:
          limits:
            cpu: "1000m"
            memory: "3Gi"
          requests:
            cpu: "50m"
            memory: "2Gi"
        ports:
        - containerPort: 8089
          name: http
          protocol: TCP
        envFrom:
          - configMapRef:
              name: canal-admin-cm
        readinessProbe:
          failureThreshold: 3
          periodSeconds: 5
          initialDelaySeconds: 10
          successThreshold: 1
          httpGet:
            port: 8089
            path: /
        livenessProbe:
          httpGet:
            port: 8089
            path: /
          initialDelaySeconds: 10
          periodSeconds: 10
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canal-admin-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: canal-admin.test.cn
    http:
      paths:
      - backend:
          service:
            name: canal-admin
            port:
              number: 8089
        path: /
        pathType: Prefix
---
# configmap
# kubectl create configmap game-config --from-file=docs/user-guide/configmap/kubectl
apiVersion: v1
kind: ConfigMap
metadata:
  name: canal-admin-cm
data:
  # replace your filename of properties configration and contents here.
  # canal-admin服务端口
  server.port: "8089"
  # canal-server注册到canal-admin时用的用户
  canal.adminUser: "admin"
  # canal-server注册到canal-admin时用的密码,注意这不是canal-admin 管理 web 的登录密码
  canal.adminPasswd: "123456"
  # 存放canal-admin数据的外部数据库配置
  spring.datasource.address: "192.168.1.221:3306"
  spring.datasource.database: "canal_manager"
  spring.datasource.username: "canal"
  spring.datasource.password: "canal"

部署 

# kubectl部署
kubectl apply -f deployment.yaml
# 查看部署结果
kubectl get svc 
NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
canal-admin             ClusterIP   10.43.215.112           8089/TCP                              4d21h
kubectl get pod -l app=canal-admin
NAME                           READY   STATUS    RESTARTS   AGE
canal-admin-795bb556d7-l2jgh   1/1     Running   0          23h
> kubectl get ingress -n tools
NAME                    CLASS   HOSTS                    ADDRESS        PORTS     AGE
canal-admin-ingress     nginx   canal-admin.test.cn     192.168.1.145   80      4d21h

部署完可以通过ingress配置的域名进行访问,如:http://canal-admin.test.cn,默认帐号密码:admin/123456

k8s 部署canal admin及server单机服务_第1张图片

 

3. 部署canal-server单机


canal-server这里部署的是单机版,不需要sql初始化,直接部署即可,注意部署中的变量传递,canal-admin password是在2.2章节中的canal.adminPasswd的密文模式

密文模式 

# 2.2配置的 canal.adminPasswd: "123456",则明文密码为:123456,密文为
# 登录任一mysql,通过select查询语句 
 select password('123456');
 password('123456')
 *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9
# 密文为查询结果去掉*的内容,如下
6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9

canal-server 部署文件
服务端口11110,用于canal-admin通信,开启自动注册到canal-admin,注册IP为canal-server-headless.svc

apiVersion: v1
kind: Service
metadata:
  name: canal-server-headless
  labels:
    app: canal-server
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - port: 11110
      name: http
      targetPort: 11110
    - name: tcp11112
      protocol: TCP
      port: 11112
      targetPort: 11112    
  selector:
    app: canal-server
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: canal-server
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - canal-server
        topologyKey: "kubernetes.io/hostname"
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: canal-server
  template:
    metadata:
      labels:
        app: canal-server
    spec:
      containers:
        - name: canal-server
          imagePullPolicy: Always
          image: canal/canal-server:v1.1.6
          resources:
            limits:
              memory: 3Gi
              cpu: 1000m
            requests:
              cpu: 50m
              memory: 2Gi
          ports:
            - containerPort: 11110
              name: http
              protocol: TCP  
            - containerPort: 11112
              name: tcp11112
              protocol: TCP  
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: SERVICE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: 'metadata.labels[''app'']'
            - name: STS_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: canal.register.ip
              value: canal-server-headless.svc
            - name: canal.admin.manager
              value: 'canal-admin:8089'
            - name: canal.admin.port
              value: '11110'
            - name: canal.admin.user
              value: 'admin'
            - name: canal.admin.passwd # 注意这里是2.2章节配置的canal-admin password密文
              value: '6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9'
            - name: canal.admin.register.auto
              value: 'true'
            - name: canal.admin.register.name
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name

部署 

# kubectl部署
kubectl apply -f deployment.yaml
kubectl get svc 
NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
canal-server-headless   ClusterIP   None                    11110/TCP,11111/TCP,11112/TCP         23h
kubectl get pod -l app=canal-server
NAME             READY   STATUS    RESTARTS   AGE
canal-server-0   1/1     Running   0          18h

部署成功后,可以在canal-admin查看注册结果及运行状态

k8s 部署canal admin及server单机服务_第2张图片

k8s 部署canal admin及server单机服务_第3张图片 

 

你可能感兴趣的:(k8s,kubernetes,数据库,java)