Flink WordCount实例讲解

    摘要：本文主要讲了一个Flink的简单的可以本地运行的wordCount程序

本文工程下载：https://github.com/appleappleapple/BigDataLearning

1、工程目录结构

2、pom文件

	4.0.0
	com.lin
	Flink-Demo
	0.0.1-SNAPSHOT
	${project.artifactId}

	
		1.6
		1.6
		UTF-8
		2.11.5
		2.11
	

	
		
			org.scala-lang
			scala-library
			${scala.version}
		

		
			org.apache.flink
			flink-scala_2.11
			1.1.2
		
		
			org.apache.flink
			flink-clients_2.11
			1.1.2
		

		
			org.apache.flink
			flink-streaming-scala_2.11
			1.1.2
		
	

	
		src/main/scala
		src/test/scala
		
			
				
				net.alchim31.maven
				scala-maven-plugin
				3.2.0
				
					
						
							compile
							testCompile
						
						
							
								-make:transitive
								-dependencyfile
								${project.build.directory}/.scala_dependencies
							
						
					
				
			
			
				org.apache.maven.plugins
				maven-surefire-plugin
				2.18.1
				
					false
					true
					
					
					
						**/*Test.*
						**/*Suite.*
					
				
			
		
	

3、代码

package com.lin.flink.demo

import org.apache.flink.api.java.utils.ParameterTool
import org.apache.flink.api.scala._
import org.apache.flink.core.fs.FileSystem.WriteMode

/**
 * 可以直接本地运行
 */
object WordCount {

  def main(args: Array[String]) {
    val env = ExecutionEnvironment.createLocalEnvironment(1)

    //从本地读取文件
    val text = env.readTextFile("D:/Java/flink-1.2.0-bin-hadoop27-scala_2.11/flink-1.2.0-bin-hadoop27-scala_2.11/flink-1.2.0/README.txt")

    //单词统计
    val counts = text.flatMap { _.toLowerCase.split("\\W+") filter { _.nonEmpty } }
      .map { (_, 1) }
      .groupBy(0)
      .sum(1)

    //输出结果
    counts.print()

    //保存结果到txt文件
    counts.writeAsText("D:/output.txt", WriteMode.OVERWRITE)
    env.execute("Scala WordCount Example")

  }
}

直接运行，输出结果如下

保存的外部文件

本文工程下载：https://github.com/appleappleapple/BigDataLearning