Mac 上搭建 Flink 1.7.2 环境并构建运行简单程序入门

准备工作

1、安装查看 Java 的版本号，推荐使用 Java 8。
2.安装 brew

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

上面方法失败的话请执行如下操作
在浏览器上进入上面的网站地址，将里面的内容复制到自己新建一个 “install.txt”里面，然后将该文件保存到任意目录，进入该目录下执行

ruby install.txt

安装 Flink

2、在 Mac OS X 上安装 Flink 是非常方便的。推荐通过 homebrew 来安装。

brew install apache-flink

3、检查安装：

kingdeMacBook-Pro:bin king$ flink --version
Version: 1.7.2, Commit ID: ceba8af

4、启动 flink

到该目录下执行/usr/local/Cellar/apache-flink/1.7.2/libexec/bin
./start-cluster.sh

Starting cluster.
Starting standalonesession daemon on host kingdeMacBook-Pro.local.
Starting taskexecutor daemon on host kingdeMacBook-Pro.local.

5、web访问

http://localhost:8081

demo

1、新建一个 maven 项目
创建一个 SocketTextStreamWordCount 文件，加入以下代码：

package com.king.learn.Flink.streaming.wordcount


import org.apache.flink.streaming.api.scala._
import org.apache.flink.util.Collector

/**
  * @Author: king
  * @Date: 2019-02-28
  * @Desc: TODO 
  */

object SocketTextStreamWordCount {
  def main(args: Array[String]): Unit = {
    //参数检查
    if (args.length != 2) {
      System.err.println("USAGE:\nSocketTextStreamWordCount  ")
    }

    val hostname = args(0)
    val port = args(1).toInt

    // get the execution environment
    val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment

    // get input data by connecting to the socket
    val stream: DataStream[String] = env.socketTextStream(hostname, port, '\n')


    val sum = stream.flatMap(new SocketTextStreamWordCount.LineSplitter).keyBy(0).sum(1)
    sum.print

    env.execute("Socket Window WordCount")
  }

  import org.apache.flink.api.common.functions.FlatMapFunction

  final class LineSplitter extends FlatMapFunction[String, (String, Integer)] {
    def flatMap(s: String, collector: Collector[(String, Integer)]): Unit = {
      val tokens = s.toLowerCase.split("\\W+")
      for (token <- tokens) {
        if (token.length > 0) collector.collect(new Tuple2[String, Integer](token, 1))
      }
    }
  }

}

maven 打包

注意我这是scala文件需要maven scala 打包插件

 <build>
        
        <sourceDirectory>src/main/scalasourceDirectory>
        <testSourceDirectory>src/test/scalatestSourceDirectory>
        
        <plugins>
            <plugin>
                <groupId>net.alchim31.mavengroupId>
                <artifactId>scala-maven-pluginartifactId>
                <version>3.2.2version>
                <executions>
                    <execution>
                        <goals>
                            <goal>compilegoal>
                            <goal>testCompilegoal>
                        goals>
                        <configuration>
                            <args>
                                
                                <arg>-dependencyfilearg>
                                <arg>${project.build.directory}/.scala_dependenciesarg>
                            args>
                        configuration>
                    execution>
                executions>
            plugin>

            
            <plugin>
                <groupId>org.apache.maven.pluginsgroupId>
                <artifactId>maven-compiler-pluginartifactId>
                <version>3.0version>
                <configuration>
                    <source>6source>
                    <target>6target>
                    <encoding>UTF-8encoding>
                configuration>
                <executions>
                    <execution>
                        <phase>compilephase>
                        <goals>
                            <goal>compilegoal>
                        goals>
                    execution>
                executions>
            plugin>


            
            <plugin>
                <groupId>net.alchim31.mavengroupId>
                <artifactId>scala-maven-pluginartifactId>
                <version>3.2.1version>
                <executions>
                    <execution>
                        <id>scala-compile-firstid>
                        <phase>process-resourcesphase>
                        <goals>
                            <goal>add-sourcegoal>
                            <goal>compilegoal>
                        goals>
                    execution>
                executions>
            plugin>


            
            <plugin>
                <groupId>org.apache.maven.pluginsgroupId>
                <artifactId>maven-shade-pluginartifactId>
                <version>2.4.3version>
                <executions>
                    <execution>
                        <phase>packagephase>
                        <goals>
                            <goal>shadegoal>
                        goals>
                        <configuration>
                            <filters>
                                <filter>
                                    <artifact>*:*artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SFexclude>
                                        <exclude>META-INF/*.DSAexclude>
                                        <exclude>META-INF/*.RSAexclude>
                                    excludes>
                                filter>
                            filters>
                            <transformers>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.confresource>
                                transformer>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>cn.itcast.rpc.MastermainClass> 
                                transformer>
                            transformers>
                        configuration>
                    execution>
                executions>
            plugin>
        plugins>
    build>

开启监听 9000 端口:

nc -l 9000

进入 flink 安装目录 bin 下执行以下命令跑程序：

./flink run -c com.king.learn.Flink.streaming.wordcount.SocketTextStreamWordCount /Volumes/mydisk/Projects/IdeaProjects/flink-learn/target/original-flink-learn-1.0-SNAPSHOT.jar 127.0.0.1 9000

注意自己的文件路径

我们可以在 webUI 中看到正在运行的程序
进入/usr/local/Cellar/apache-flink/1.7.2/libexec/log

tail -f flink-king-taskexecutor-0-kingdeMacBook-Pro.local.out