Java实现GroupBy/分组TopN功能
介绍
在Java 8 的Lambda（stream）之前，要在Java代码中实现类似SQL中的group by分组聚合功能，还是比较困难的。这之前Java对函数式编程支持不是很好，Scala则把函数式编程发挥到了机制，实现一个group by聚合对Scala来说就是几行代码的事情：
val birds = List("Golden Eagle","Gyrfalcon", "American Robin", "Mountain BlueBird", "Mountain-Hawk Eagle")
val groupByFirstLetter = birds.groupby(_.charAt(0))输出：
Map(M -> List(Mountain BlueBird, Mountain-Hawk Eagle), G -> List(Golden Eagle, Gyrfalcon),
A -> List(American Robin))Java也有一些第三方的函数库来支持，例如Guava的Function，以及functional java这样的库。 但总的来说，内存对Java集合进行GroupBy ，OrderBy， Limit等TopN操作还是比较繁琐。本文实现一个简单的group功能，支持自定义key以及聚合函数，通过简单的几个类，可以实现SQL都比较难实现的先分组，然后组内排序，最后取组内TopN。
源码可以在这里下载；
实现
假设我们有这样一个Person类：
package me.lin;
class Person {
private String name;
private int age;
private double salary;
public Person(String name, int age, double salary) {
super();
this.name = name;
this.age = age;
this.salary = salary;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public double getSalary() {
return salary;
}
public void setSalary(double salary) {
this.salary = salary;
}
public String getNameAndAge() {
return this.getName() + "-" + this.getAge();
}
@Override
public String toString() {
return "Person [name=" + name + ", age=" + age + ", salary=" + salary
+ "]";
}
}对于一个Person的List，想要根据年龄进行统计，取第一个值，取salary最高值等。实现如下：
聚合操作
定义一个聚合接口，用于对分组后的元素进行聚合操作，类比到MySQL中的count(*) 、sum():
package me.lin;
import java.util.List;
/**
*
* 聚合操作
*
* Created by Brandon on 2016/7/21.
*/
public interface Aggregator {
/**
* 每一组的聚合操作
*
* @param key 组别标识key
* @param values 属于该组的元素集合
* @return
*/
Object aggregate(Object key , List values);
}
我们实现几个聚合操作，更复杂的操作支持完全可以自己定义。
CountAggragator:
package me.lin;
import java.util.List;
/**
*
* 计数聚合操作
*
* Created by Brandon on 2016/7/21.
*/
public class CountAggregator implements Aggregator {
@Override
public Object aggregate(Object key, List values) {
return values.size();
}
}
FisrtAggregator:
package me.lin;
import java.util.List;
/**
*
* 取第一个元素
*
* Created by Brandon on 2016/7/21.
*/
public class FirstAggregator implements Aggregator {
@Override
public Object aggregate(Object key, List values) {
if ( values.size() >= 1) {
return values.get( 0 );
}else {
return null;
}
}
}
TopNAggregator:
package me.lin;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
/**
*
* 取每组TopN
*
* Created by Brandon on 2016/7/21.
*/
public class TopNAggregator implements Aggregator {
private Comparator comparator;
private int limit;
public TopNAggregator(Comparator comparator, int limit) {
this.limit = limit;
this.comparator = comparator;
}
@Override
public Object aggregate(Object key, List values) {
if (values == null || values.size() == 0) {
return null;
}
ArrayList copy = new ArrayList<>( values );
Collections.sort(copy, comparator);
int size = values.size();
int toIndex = Math.min(limit, size);
return copy.subList(0, toIndex);
}
} 分组实现
接下来是分组实现，简单起见，采用工具类实现：
package me.lin;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
/**
* Collection分组工具类
*/
public class GroupUtils {
/**
* 分组聚合
*
* @param listToDeal 待分组的数据，相当于SQL中的原始表
* @param clazz 带分组数据元素类型
* @param groupBy 分组的属性名称
* @param aggregatorMap 聚合器，key为聚合器名称，作为返回结果中聚合值map中的key
* @param 元素类型Class
* @return
* @throws NoSuchFieldException
* @throws SecurityException
* @throws IllegalArgumentException
* @throws IllegalAccessException
*/
public static Map> groupByProperty(
Collection listToDeal, Class clazz, String groupBy,
Map> aggregatorMap) throws NoSuchFieldException,
SecurityException, IllegalArgumentException, IllegalAccessException {
Map> groupResult = new HashMap>();
for (T ele : listToDeal) {
Field field = clazz.getDeclaredField(groupBy);
field.setAccessible(true);
Object key = field.get(ele);
if (!groupResult.containsKey(key)) {
groupResult.put(key, new ArrayList());
}
groupResult.get(key).add(ele);
}
return invokeAggregators(groupResult, aggregatorMap);
}
public static Map> groupByMethod(
Collection listToDeal, Class clazz, String groupByMethodName,
Map> aggregatorMap) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
Map> groupResult = new HashMap>();
for (T ele : listToDeal) {
Method groupByMenthod = clazz.getDeclaredMethod(groupByMethodName);
groupByMenthod.setAccessible(true);
Object key = groupByMenthod.invoke(ele);
if (!groupResult.containsKey(key)) {
groupResult.put(key, new ArrayList());
}
groupResult.get(key).add(ele);
}
return invokeAggregators(groupResult, aggregatorMap);
}
private static Map> invokeAggregators(Map> groupResult, Map> aggregatorMap) {
Map> aggResults = new HashMap<>();
for (Object key : groupResult.keySet()) {
Collection group = groupResult.get(key);
Map aggValues = doInvokeAggregators(key, group, aggregatorMap);
if (aggValues != null && aggValues.size() > 0) {
aggResults.put(key, aggValues);
}
}
return aggResults;
}
private static Map doInvokeAggregators(Object key, Collection group, Map> aggregatorMap) {
Map aggResults = new HashMap();
if (group != null && group.size() > 0) {
// 调用当前key的每一个聚合函数
for (String aggKey : aggregatorMap.keySet()) {
Aggregator aggregator = aggregatorMap.get(aggKey);
Object aggResult = aggregator.aggregate(key, Collections.unmodifiableList(new ArrayList(group)));
aggResults.put(aggKey, aggResult);
}
}
return aggResults;
}
} 上述代码中，分组的key可以指定元素的属性，也可以指定元素的方法，通过自己实现复杂方法和聚合函数，可以实现很强大的分组功能。
测试
根据属性分组
下面测试一下根据属性分组：
package me.lin;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class GroupByPropertyTest {
public static void main(String[] args) throws NoSuchFieldException,
SecurityException, IllegalArgumentException, IllegalAccessException {
List persons = new ArrayList<>();
persons.add(new Person("Brandon", 15, 5000));
persons.add(new Person("Braney", 15, 15000));
persons.add(new Person("Jack", 10, 5000));
persons.add(new Person("Robin", 10, 500000));
persons.add(new Person("Tony", 10, 1400000));
Map> aggregatorMap = new HashMap<>();
aggregatorMap.put("count", new CountAggregator());
aggregatorMap.put("first", new FirstAggregator());
Comparator comparator = new Comparator() {
public int compare(final Person o1, final Person o2) {
double diff = o1.getSalary() - o2.getSalary();
if (diff == 0) {
return 0;
}
return diff > 0 ? -1 : 1;
}
};
aggregatorMap.put("top2", new TopNAggregator( comparator , 2 ));
Map> aggResults = GroupUtils.groupByProperty(persons, Person.class, "age", aggregatorMap);
for (Object key : aggResults.keySet()) {
System.out.println("Key:" + key);
Map results = aggResults.get(key);
for (String aggKey : results.keySet()) {
System.out.println(" aggkey->" + results.get(aggKey));
}
}
}
} 输出结果：
Key:10
aggkey->3
aggkey->Person [name=Jack, age=10, salary=5000.0]
aggkey->[Person [name=Tony, age=10, salary=1400000.0], Person [name=Robin, age=10, salary=500000.0]]
Key:15
aggkey->2
aggkey->Person [name=Brandon, age=15, salary=5000.0]
aggkey->[Person [name=Braney, age=15, salary=15000.0], Person [name=Brandon, age=15, salary=5000.0]]根据方法返回值分组
测试根据方法返回值分组：
package me.lin;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class GroupByMethodTest {
public static void main(String[] args) throws Exception {
List persons = new ArrayList<>();
persons.add(new Person("Brandon", 15, 5000));
persons.add(new Person("Brandon", 15, 15000));
persons.add(new Person("Jack", 10, 5000));
persons.add(new Person("Robin", 10, 500000));
persons.add(new Person("Tony", 10, 1400000));
Map> aggregatorMap = new HashMap<>();
aggregatorMap.put("count", new CountAggregator());
aggregatorMap.put("first", new FirstAggregator());
Comparator comparator = new Comparator() {
public int compare(final Person o1, final Person o2) {
double diff = o1.getSalary() - o2.getSalary();
if (diff == 0) {
return 0;
}
return diff > 0 ? -1 : 1;
}
};
aggregatorMap.put("top2", new TopNAggregator(comparator, 2));
Map> aggResults = GroupUtils.groupByMethod(persons, Person.class, "getNameAndAge", aggregatorMap);
for (Object key : aggResults.keySet()) {
System.out.println("Key:" + key);
Map results = aggResults.get(key);
for (String aggKey : results.keySet()) {
System.out.println(" " + aggKey + "->" + results.get(aggKey));
}
}
}
} 测试结果：
Key:Robin-10
count->1
first->Person [name=Robin, age=10, salary=500000.0]
top2->[Person [name=Robin, age=10, salary=500000.0]]
Key:Jack-10
count->1
first->Person [name=Jack, age=10, salary=5000.0]
top2->[Person [name=Jack, age=10, salary=5000.0]]
Key:Tony-10
count->1
first->Person [name=Tony, age=10, salary=1400000.0]
top2->[Person [name=Tony, age=10, salary=1400000.0]]
Key:Brandon-15
count->2
first->Person [name=Brandon, age=15, salary=5000.0]
top2->[Person [name=Brandon, age=15, salary=15000.0], Person [name=Brandon, age=15, salary=5000.0]]以上就是GroupBy的简单实现，如果问题，欢迎指出。
有兴趣可以加一下854630135这个群去交流一下噢
欢迎交流。
转载于:https://blog.51cto.com/13954634/2313665