Hadoop(19)-MapReduce框架原理-Combiner合并

1. Combiner概述

 

 2. 自定义Combiner实现步骤

1). 定义一个Combiner继承Reducer,重写reduce方法

public class WordcountCombiner extends Reducer{

    @Override
    protected void reduce(Text key, Iterable values,Context context) throws IOException, InterruptedException {

        // 1 汇总操作
        int count = 0;
        for(IntWritable v :values){
            count += v.get();
        }

        // 2 写出
        context.write(key, new IntWritable(count));
    }
}

2). 在Driver类中添加设置

job.setCombinerClass(WordcountCombiner.class);

 

效果

 

 

 

转载于:https://www.cnblogs.com/duoduotouhenying/p/10110510.html