Hadoop Problem

Reduce fetch map data ,but OOM

2015-11-26 15:02:18,687 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager:
memoryLimit=1 336 252 800, maxSingleShuffleLimit=133625280, mergeThreshold=881926912, ioSortFactor=10, memToMemMergeOutputsThreshold=10

memoryLimit : 1.2G
maxSingleShuffleLimit : 0.12G
mergeThreshold : 0.841 G
ioSortFactor : 10
memToMemMergeOutputsThreshold : 10

方法已经同步，排除多线程可能

1. size < maxSingleShuffleLimit : 没有120M，不需要DiskOutput
2. current used size < mergeThreshold : 当前使用的Mem < 841M ,merge not triggered
3. 当前剩余的内存 (2G - 841M) 是足够120M数据的mem merge的