Flink实战三 Flink的安装与部署及使用

安装Flink1.9

1、准备：jdk 1.8

     SSH免密登录

     三台以上机器（我准备了五台）：centos7-1、centos7-2、centos7-3、centos7-4、centos7-5

     zookeeper: centos7-3、centos7-4、centos7-5

     ...

2、从官网下载flink：https://flink.apache.org/downloads.html

3、把文件tar开

$ tar -zxvf flink-1.9.0-bin-scala_2.11.tgz

4、 修改配置

$ vim /conf/slaves
#添加一下内容
centos7-2
centos7-3
centos7-4
centos7-5

$ vim /conf/flink-conf.yaml
jobmanager.rpc.address: 192.168.2.141

5、启动Flink集群

$ bin/start-cluster.sh

6、验证

# centos7-1机器
$ jps
18376 Jps
14922 StandaloneSessionClusterEntrypoint

# 其他机器
$ jps
9899 Jps
9821 TaskManagerRunner

7、Flink的web ui

http://localhost:8081     （localhost替换成实际的ip）

 

JobManager高可用配置（HA）

1、编辑masters文件

$ vim /conf/masters
centos7-1:8081
centos7-2:8081
centos7-3:8081

2、编辑flink-conf.yaml

$ vim /conf/flink-conf.yaml
high-availability: zookeeper
high-availability.storageDir: hdfs://centos7-1:8020/flink/ha/
high-availability.zookeeper.quorum: centos7-3:2181,centos7-4:2181,centos7-5:2181
high-availability.zookeeper.path.root: /flink

3、编辑zoo.cfg

$ vim /conf/zoo.cfg
server.1=centos7-3:2888:3888
server.2=centos7-4:2888:3888
server.3=centos7-5:2888:3888

4、分发到其他机器

$ scp -r flink-1.9.0 hadoop@centos7-2:/home/hadoop/
$ scp -r flink-1.9.0 hadoop@centos7-3:/home/hadoop/
$ scp -r flink-1.9.0 hadoop@centos7-4:/home/hadoop/
$ scp -r flink-1.9.0 hadoop@centos7-5:/home/hadoop/

5、启动集群

$ ./bin/start-cluster.sh
Starting HA cluster with 3 masters.
Starting standalonesession daemon on host centos7-1.
Starting standalonesession daemon on host centos7-2.
Starting standalonesession daemon on host centos7-3.
Starting taskexecutor daemon on host centos7-2.
Starting taskexecutor daemon on host centos7-3.
Starting taskexecutor daemon on host centos7-4.
Starting taskexecutor daemon on host centos7-5.

6、查看进程

#masters文件配置的机器：StandaloneSessionClusterEntrypoint
#slaves文件配置的机器：TaskManagerRunner
$ jps 
15908 StandaloneSessionClusterEntrypoint
16151 TaskManagerRunner

7、Flink的web ui

http://localhost:8081    （localhost替换成实际的ip）

 

简单使用