Flink docker-compose 单机版 安装教程

介绍

运行模式：

1. Flink Session Cluster
2. Flink Job Cluster
3. Flink Application Cluster

三种集群模式：

1. standalone
2. kubernetes
3. YARN

本文用 配置最简单的 Session Cluster +docker-compose 做演示

配置安装

设置版本环境变量 对应的版本可以看这里

# windows power shell 设置环境 flink 版本环境变量 
$Env:FLINK_DOCKER_IMAGE_NAME="flink:1.14.5-scala_2.12-java8"

编写 docker-compose.yml 带 配置文件挂载

version: "2.1"
services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
    volumes:
      - E:\docker-mapping\Flink\conf\job\flink-conf.yaml:/opt/flink/conf/flink-conf.yaml

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager
    volumes:
      - E:\docker-mapping\Flink\conf\task\flink-conf.yaml:/opt/flink/conf/flink-conf.yaml

把配置文件放在 本地挂载的目录flink-conf.yaml

jobmanager.rpc.address: jobmanager
jobmanager.rpc.port: 6123
jobmanager.memory.process.size: 4096m
taskmanager.memory.process.size: 16384m
taskmanager.numberOfTaskSlots: 10
parallelism.default: 1
jobmanager.execution.failover-strategy: region
blob.server.port: 6124
query.server.port: 6125

启动

docker-compose.exe -f E:\docker-mapping\Flink\docker-compose.yml up -d

最后

浏览器打开 Apache Flink Dashboard

http://localhost:8081/#/overview

如果 需要增加 fink cdc jar 就去 github 下载对应版本的放到 docker flink classpath 下的 lib 目录

 docker cp C:\Users\Administrator\Downloads\flink-sql-connector-mysql-cdc-2.2.1.jar f85b707928af794ac98ebbf57642f7303b332e7f80cbae71095e07544e0a22a9:/opt/flink/lib