Saprk aggregateByKey操作示例

aggregateByKey(zeroValue)(seqOp, combOp, [numTasks])

When called on a dataset of (K, V) pairs, returns a dataset of (K, U) pairs where the values for each key are aggregated using the given combine functions and a neutral "zero" value. Allows an aggregated value type that is different than the input value type, while avoiding unnecessary allocations. Like in groupByKey, the number of reduce tasks is configurable through an optional second argument.

aggreateByKey(zeroValue: U)(seqOp: (U, T)=> U, combOp: (U, U) =>U) 和reduceByKey的不同在于，

reduceByKey输入输出都是(K, V)，

aggreateByKey输出是(K,U)，可以不同于输入(K, V) ，

aggreateByKey的三个参数：
zeroValue: U，初始值，比如空列表{} ；
seqOp: (U,T)=> U，seq操作符，描述如何将T合并入U，比如如何将item合并到列表 ；
combOp: (U,U) =>U，comb操作符，描述如果合并两个U，比如合并两个列表 ；
所以aggreateByKey可以看成更高抽象的，更灵活的reduce或group 。
val z = sc.parallelize(List(1,2,3,4,5,6), 2)
z.aggreate(0)(math.max(_, _), _ + _)
res0: Int = 9
val z = sc.parallelize(List((1, 3), (1, 2), (1, 4), (2, 3)))
z.aggregateByKey(0)(math.max(_, _), _ + _)
res1: Array[(Int, Int)] = Array((2,3), (1,9))