在用mybatis作为持久层框架时，有时候会有需要进行批量增删改的操作。
百度了一下，大致有两种方法，一种是拼接SQL的方式。类似这样：
insert into t_project ( projectid,productid ) values ( #{item.projectid,jdbcType=CHAR},#{item.productid,jdbcType=CHAR} )
这种方法局限性太多，sql长度有限制，clob类型不能这样使用，貌似只能用于insert语句。所以完全不考虑。
第二种是用mybatis提供的BatchExecutor进行批处理操作。类似这样：
SqlSession session =sessionFactory.openSession(ExecutorType.BATCH,false);
for(int i=0;i<10;i++){
session.update("com.com.hcd.mybatis.interfaces.CommonMapper.update",getParam());
}
session.flushStatements();
session.commit();
这里的"com.com.hcd.mybatis.interfaces.CommonMapper.update"就是xml配置里namespace加
如果要让spring控制批处理事务，那只能所有的事务都使用批处理模式，即SqlSessionTemplate设置executorType为batch，但是这样会有问题，如：使用数据库自增主键则无法获得返回的主键，我这边的系统ID都是在java内存生成uuid后再插入到数据库的，所以这个没有影响；增删改操作无法返回影响行数，因为batchExecutor中update方法默认放回的是一个int常量。(其原因都是因为此时sql尚未在数据库执行，所以无法获得这些信息)。
所以，这个方法也不好。根本原因是mybatis在创建SqlSession的时候，去实例化executor时，就决定了执行sql的方式是不是batch方式。在实例化SImpleExector的时候，则表明该事务的所有增删改方法会立刻在数据库执行该语句，实例化BatchExector的时候，则表明该事务的所有增删改方法都是 以batch方式，不会立刻在数据库执行。即mybatis框架本身限制了你不能灵活的在一次事务中选择是否要以批处理的方式执行。
所以，我们就会想着去拓展mybatis框架，我这边mybatis版本是3.2.2。如果BatchExector在执行update方法后立刻执行doFlushStatements，则相当于SImpleExector下执行update方法。即batchExector工作机制已经满足我们的需求，想法是扩展新的执行器继承BatchExector，重写doUpdate方法，判断是否要以批处理方式执行，对应是否调用doFlushStatements。
下面是做了一个简单的测试，测试要点是想看用批处理做频繁的更新操作和非批处理做频繁的更新操作，效率差多少，BatchExector执行addBatch后立刻doFlushStatements，和SImpleExector执行update方法是否存在性能的差距。测试用例如下：
package com.hcd.mybatis.demo;
import org.apache.ibatis.session.ExecutorType;
import org.apache.ibatis.session.SqlSession;
import org.apache.ibatis.session.SqlSessionFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.util.HashMap;
import java.util.Map;
/**
* Created by cd_huang on 2017/8/21.
*/
@Component
public class BatchMybatisTest {
@Autowired
private SqlSessionFactory sessionFactory;
public void test(){
int group =26;
long times[][] =new long[group][3];
for(int i=0;i getParam(){
Map param=new HashMap();
return param;
}
}
执行结果如下：
Batch - Executor - Batch - Execute: 123 ms Batch - Executor - Simple - Execute: 271 ms Simple - Executor - Simple - Execute: 197 ms Batch - Executor - Batch - Execute: 181 ms Batch - Executor - Simple - Execute: 524 ms Simple - Executor - Simple - Execute: 596 ms Batch - Executor - Batch - Execute: 238 ms Batch - Executor - Simple - Execute: 664 ms Simple - Executor - Simple - Execute: 523 ms Batch - Executor - Batch - Execute: 370 ms Batch - Executor - Simple - Execute: 747 ms Simple - Executor - Simple - Execute: 814 ms Batch - Executor - Batch - Execute: 311 ms Batch - Executor - Simple - Execute: 688 ms Simple - Executor - Simple - Execute: 661 ms Batch - Executor - Batch - Execute: 357 ms Batch - Executor - Simple - Execute: 799 ms Simple - Executor - Simple - Execute: 765 ms Batch - Executor - Batch - Execute: 431 ms Batch - Executor - Simple - Execute: 917 ms Simple - Executor - Simple - Execute: 878 ms Batch - Executor - Batch - Execute: 471 ms Batch - Executor - Simple - Execute: 991 ms Simple - Executor - Simple - Execute: 848 ms Batch - Executor - Batch - Execute: 339 ms Batch - Executor - Simple - Execute: 1004 ms Simple - Executor - Simple - Execute: 1087 ms Batch - Executor - Batch - Execute: 820 ms Batch - Executor - Simple - Execute: 1082 ms Simple - Executor - Simple - Execute: 1443 ms Batch - Executor - Batch - Execute: 746 ms Batch - Executor - Simple - Execute: 1535 ms Simple - Executor - Simple - Execute: 1514 ms Batch - Executor - Batch - Execute: 875 ms Batch - Executor - Simple - Execute: 1696 ms Simple - Executor - Simple - Execute: 1677 ms Batch - Executor - Batch - Execute: 810 ms Batch - Executor - Simple - Execute: 1626 ms Simple - Executor - Simple - Execute: 1576 ms Batch - Executor - Batch - Execute: 748 ms Batch - Executor - Simple - Execute: 1506 ms Simple - Executor - Simple - Execute: 1375 ms Batch - Executor - Batch - Execute: 751 ms Batch - Executor - Simple - Execute: 1544 ms Simple - Executor - Simple - Execute: 1387 ms Batch - Executor - Batch - Execute: 689 ms Batch - Executor - Simple - Execute: 1536 ms Simple - Executor - Simple - Execute: 1490 ms Batch - Executor - Batch - Execute: 772 ms Batch - Executor - Simple - Execute: 1995 ms Simple - Executor - Simple - Execute: 2242 ms Batch - Executor - Batch - Execute: 1143 ms Batch - Executor - Simple - Execute: 2530 ms Simple - Executor - Simple - Execute: 2708 ms Batch - Executor - Batch - Execute: 1705 ms Batch - Executor - Simple - Execute: 2600 ms Simple - Executor - Simple - Execute: 2745 ms Batch - Executor - Batch - Execute: 1427 ms Batch - Executor - Simple - Execute: 2516 ms Simple - Executor - Simple - Execute: 2574 ms Batch - Executor - Batch - Execute: 1191 ms Batch - Executor - Simple - Execute: 2157 ms Simple - Executor - Simple - Execute: 2142 ms Batch - Executor - Batch - Execute: 708 ms Batch - Executor - Simple - Execute: 2170 ms Simple - Executor - Simple - Execute: 2173 ms Batch - Executor - Batch - Execute: 818 ms Batch - Executor - Simple - Execute: 2415 ms Simple - Executor - Simple - Execute: 2419 ms Batch - Executor - Batch - Execute: 1140 ms Batch - Executor - Simple - Execute: 2570 ms Simple - Executor - Simple - Execute: 2244 ms Batch - Executor - Batch - Execute: 785 ms Batch - Executor - Simple - Execute: 2205 ms Simple - Executor - Simple - Execute: 2181 ms可以看出来，批处理模式下，进行大量update操作时，性能差不多可以提升60-70%。且BatchExector执行addBatch后立刻doFlushStatements，和SImpleExector执行update方法消耗的时候非常接近。
测试表明了我们拓展mybatis的框架思路是有可行性的。下面我直接贴代码介绍下我如何拓展mybatis框架实现灵活批处理。
扩展入口是利用mybatis提供的拦截器机制，实现自定义的代理执行器对象。需要在mybatis-config.xml里配置
ProxyExecutorInterceptor：
package com.hcd.mybatis.exector;
import org.apache.ibatis.executor.Executor;
import org.apache.ibatis.plugin.Interceptor;
import org.apache.ibatis.plugin.Invocation;
import java.lang.reflect.Proxy;
import java.util.Properties;
/**
* 使用插件生成代理执行器
* Created by cd_huang on 2017/8/22.
*/
public class ProxyExecutorInterceptor implements Interceptor {
@Override
public Object intercept(Invocation invocation) throws Throwable {
return null;
}
@Override
public Object plugin(Object target) {
if (target instanceof Executor == false) {
return target;
}
return Proxy.newProxyInstance(
target.getClass().getClassLoader(),
target.getClass().getInterfaces(),
new ProxyExecutorHandler((Executor)target));
}
@Override
public void setProperties(Properties properties) {
}
} 这里没用默认的Plugin.wrap()来代理对象，而是用我们自定义的ProxyExecutorHandler来进行代理控制。
ProxyExecutorHandler：
package com.hcd.mybatis.exector;
import com.hcd.common.SpringContextUtil;
import org.apache.commons.lang3.reflect.FieldUtils;
import org.apache.ibatis.executor.*;
import org.apache.ibatis.plugin.Plugin;
import org.apache.ibatis.reflection.ReflectionException;
import org.apache.ibatis.session.Configuration;
import org.apache.ibatis.session.ExecutorType;
import org.apache.ibatis.session.SqlSessionFactory;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
/**
* 代理拦截器，控制ComplexExecutor的初始化时机
* Created by cd_huang on 2017/8/23.
*/
public class ProxyExecutorHandler implements InvocationHandler {
private Executor target;
private Executor realTarget;
private boolean isInit = false;
public ProxyExecutorHandler(Executor target) {
this.target = target;
MybatisExecutorContext.bindExecutor(this);
if (MybatisExecutorContext.initComplexExecutorImmediately) {
init();
}
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
if (!isInit && MybatisExecutorContext.isOpenExecutorMode()) {
init();
}
try {
return method.invoke(this.target, args);
} finally {
String methodName = method.getName();
if ("commit".equals(methodName) || "rollback".equals(methodName) || "close".equals(methodName)) {
MybatisExecutorContext.clean();
}
}
}
public void init() {
this.target = initComplexExecutor(this.target);
isInit = true;
}
/**
* 初始化ComplexExecutor
*
* @param target
* @return
*/
public Executor initComplexExecutor(Executor target) {
Executor initialTarget = target;
Object[] result = ProxyExecutorHandler.getLastPluginFromJdkProxy(target);
Object parentPlugin = result[0];
target = (Executor) result[1];
if (target instanceof ComplexExecutor) {
realTarget =target;
return initialTarget;
}
if (target instanceof BaseExecutor) {
ComplexExecutor newTarget = new ComplexExecutor(ProxyExecutorHandler.getConfiguration(), target.getTransaction(), getOriginalExecutorType((Executor)target));
realTarget =newTarget;
if (parentPlugin == null) {
return newTarget;
} else {
//替换掉Plugin的target属性
try {
Field targetField = FieldUtils.getField(Plugin.class, "target", true);
FieldUtils.writeField(targetField, parentPlugin, newTarget, true);
return initialTarget;
} catch (Throwable e) {
throw new ReflectionException("replace property 'target' of Plugin error ！", e);
}
}
}
//替换掉CachingExecutor的delegate属性
if (target instanceof CachingExecutor) {
try {
Field delegateField = FieldUtils.getField(CachingExecutor.class, "delegate", true);
Object delegate = FieldUtils.readField(delegateField, target, true);
if (delegate instanceof BaseExecutor) {
Executor newDelegate = new ComplexExecutor(ProxyExecutorHandler.getConfiguration(), ((BaseExecutor) delegate).getTransaction(), getOriginalExecutorType((Executor)delegate));
realTarget =newDelegate;
FieldUtils.writeField(delegateField, target, newDelegate, true);
return initialTarget;
}
} catch (Throwable e) {
throw new ReflectionException("replace property 'delegate' of CachingExecutor error ！", e);
}
}
throw new ExecutorException("init ComplexExecutor error ！");
}
/**
* 获得jdk代理里最里层的一个Plugin对象
*
* @param proxy
* @return 返回 最里层的一个Plugin对象 和 Plugin的target属性
*/
public static Object[] getLastPluginFromJdkProxy(Object proxy) {
if (proxy instanceof Proxy) {
InvocationHandler h = Proxy.getInvocationHandler(proxy);
Object target;
try {
Field targetField = FieldUtils.getField(Plugin.class, "target", true);
target = FieldUtils.readField(targetField, h, true);
} catch (Throwable e) {
throw new ReflectionException("get property 'target' of Plugin error ！", e);
}
if (target instanceof Proxy) {
return ProxyExecutorHandler.getLastPluginFromJdkProxy(target);
} else {
return new Object[]{h, target};
}
} else {
return new Object[]{null, proxy};
}
}
private static Configuration configuration;
public static Configuration getConfiguration() {
if (configuration == null) {
//从spring上下文中拿到SqlSessionFactory，从而拿到configuration
configuration = SpringContextUtil.getBean(SqlSessionFactory.class).getConfiguration();
}
return configuration;
}
public ExecutorType getOriginalExecutorType() {
if(realTarget!=null){
return getOriginalExecutorType(realTarget);
}
throw new ExecutorException("ComplexExecutor not init !");
}
public static ExecutorType getOriginalExecutorType(Executor executor) {
if(executor instanceof ComplexExecutor){
return ((ComplexExecutor)executor).getOriginalExecutorType();
}
if (BatchExecutor.class.getName().equals(executor.getClass().getName())) {
return ExecutorType.BATCH;
} else {
return ExecutorType.SIMPLE;
}
}
public Executor getRealTarget(){
return realTarget;
}
public Executor getTarget() {
return target;
}
public void setTarget(Executor target) {
this.target = target;
}
}
主要是控制把SimpleExector或BatchExector替换成我们自定义的执行器对象ComplexExecutor。
ComplexExecutor：
package com.hcd.mybatis.exector;
import org.apache.ibatis.executor.BatchExecutor;
import org.apache.ibatis.executor.BatchResult;
import org.apache.ibatis.mapping.MappedStatement;
import org.apache.ibatis.session.Configuration;
import org.apache.ibatis.session.ExecutorType;
import org.apache.ibatis.transaction.Transaction;
import java.sql.SQLException;
import java.util.List;
/**
* 扩展mybatis支持批处理和非批处理两种模式的复合执行器
* Created by cd_huang on 2017/8/22.
*/
public class ComplexExecutor extends BatchExecutor{
private ExecutorType originalExecutorType;
public ComplexExecutor(Configuration configuration, Transaction transaction,ExecutorType originalExecutorType) {
super(configuration, transaction);
this.originalExecutorType =originalExecutorType;
}
public int doUpdate(MappedStatement ms, Object parameterObject) throws SQLException {
int result =super.doUpdate(ms,parameterObject);
if(!MybatisExecutorContext.isBatchExecutorMode()){
List results=this.doFlushStatements(false);
return results.get(0).getUpdateCounts()[0];
}
return result;
}
public List doFlushStatements(boolean isRollback) throws SQLException {
List results =super.doFlushStatements(isRollback);
MybatisExecutorContext.getCheckBatchResultHook().checkBatchResult(results);
return results;
}
public ExecutorType getOriginalExecutorType() {
return originalExecutorType;
}
}
该执行器重写了doUpdate()方法，判断是否要使用批处理模式，从而判断是否执行doFlushStatements。 判断是否要执行批处理模式，是在MybatisExecutorContext类中，利用线程变量，来主动设置的。MybatisExecutorContext：