k8s部署airflow+mysql

k8s启动airflow + mysql

创建命名空间脚本create_shjujubu_namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: shujubu
  labels:
    name: shujubu

执行文件创建 shujubu命名空间：

kubectl create -f  create_shjujubu_namespace.yaml

定义mysql的pod文件 mysql-rc.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  namespace: shujubu
  labels:
    name: mysql
spec:
  containers:
  - name: mysql
    image: ccr.ccs.tencentyun.com/shujubu/luban-mysql:1.0.8
    imagePullPolicy: IfNotPresent
    ports:
    - containerPort: 3306
    env:
    - name: MYSQL_SERVICE_HOST
      value: "mysql"
    - name: MYSQL_SERVICE_PORT
      value: "3306"
    - name: MYSQL_ROOT_PASSWORD
      value: '123456'
    - name: MYSQL_DATABASE
      value: 'airflow'
    - name: TZ
      value: 'Asia/Shanghai'
    - name: LANG
      value: 'C.UTF-8'

定义mysql的service文件 mysql-np.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    name: mysql
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306

启动pod命令：

kubectl create -f mysql-pod.yaml -n shujubu

启动service命令：

kubectl create -f mysql-np.yaml -n shujubu

然后就可以在同一k8s集群中其他pod中通过uri: mysql+mysqldb://root:123456@mysql:3306/airflow 连接到这个mysql啦

mysql服务启动完成，接下来启动airflow。首先定义airflow-pod.yaml文件：

apiVersion: v1
kind: Pod
metadata:
  name: luban-airflow
  namespace: shujubu
  labels:
    name: luban-airflow
spec:
  containers:
  - name: worker
    image: ccr.ccs.tencentyun.com/shujubu/luban-airflow:1.0.8
    imagePullPolicy: IfNotPresent
    ports:
    - containerPort: 10000
    command: ["/bin/sh"]
    args: ["-c", "airflow initdb;airflow webserver -p 10000 -D;airflow scheduler -D;"]
    volumeMounts:
    - mountPath: /usr/local/airflow/logs
      name: airflow-log
    env:
    - name: AIRFLOW__CORE__FERNET_KEY
      value: "TSvxYPRwt5hVLCQPQzDdL7lp-jt3oZOocJE4G1UsocU="
      #value: echo $(python -c "from cryptography.fernet import Fernet; FERNET_KEY = Fernet.generate_key().decode(); print(FERNET_KEY)")  
    - name: AIRFLOW__CORE__SQL_ALCHEMY_CONN
      value: "mysql+mysqldb://root:123456@mysql:3306/airflow"

  restartPolicy: Never

  volumes:                                                                                                                                                                                           
  - name: airflow-log
    hostPath:
      path: /home/users/qix/logs/airflow
      

然后定义airflow-np.yaml文件暴露airflow服务：

apiVersion: v1
kind: Service
metadata:
  name: luban-airflow
spec:
  selector:
    name: luban-airflow
  type: NodePort
  ports:
    - protocol: TCP
      port: 10000
      targetPort: 10000
      nodePort: 30001

大功告成！访问http://主机IP:30001/admin即可访问airflow服务啦：