Flink系列：单词统计Demo（批处理、流式处理）

目录

批处理demo

流式处理demo

---

批处理demo

package csdn.xdoctorx;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.operators.DataSource;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;
//import scala.Tuple2;


public class WordCountReadFromFile {

    public static void main(String[] args) throws Exception {
        String inputFile = "D:\\tmp\\input.txt" ;
        String outputFile = "D:\\tmp\\output.txt" ;

        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment() ;

        DataSource text = env.readTextFile(inputFile);
        DataSet> counts = text.flatMap(new WordCountMap()).groupBy(0).sum(1) ;
        counts.writeAsCsv(outputFile,"\n","").setParallelism(1);
        env.execute("batch word count!") ;
    }
}

class WordCountMap implements FlatMapFunction> {
    @Override
    public void flatMap(String value, Collector> collector) throws Exception {
        String[] words = value.split("\\W+") ;
        for (String word:words) {
            collector.collect(new Tuple2(word,1));
        }
    }
}

 

  

 

流式处理demo

Flink系列：IDEA创建flink项目开发环境（附完整WordCount例子）

Flink系列：WindowFunction事件Time时间乱序处理Demo例子