最近手头自己基于springboot2.0+mybatis搭建的开发框架遇到一个需求,需要在项目中引入多数据源,于是网上搜索了一把,搜到最多的方案是(注入多个DataSource,然后注入多个SqlSessionFactory,SqlSessionTemplate,并且在Mybatis的MapperScan包扫描注解上指定不同包对应的SqlSessionFactory),但是此种方案有几个缺点
基于以上原因,在查阅了多方资料+自己摸索后,使用成功实现了基于AbstractRoutingDataSource动态数据源切换的多数据源整合,并且完美解决了以上问题
注:以下代码是从项目中改名剥离出来的,若存在些许笔误,读者自行判断修改吧
Spring动态数据源切换主要依赖于其提供的AbstractRoutingDataSource,他是一个数据源路由,我们的所有数据源都会被注册进这个路由,并且其本身也继承DataSource,所以也实现了数据源该有的功能,我们需要实现这个抽象类,于是新建类MyDynamicDataSource,重写determineCurrentLookupKey,这个方法让我们自己决定当前该使用哪个数据源,然后返回当前应该使用的数据源名称,如果返回null则使用默认数据源,所以我们提供了allowSwitch,beginSwitch和tryBeginSwitch,endSwitch作为手动切换数据源的方法,在begin和end之间的操作会被指定为特定数据源,不在这个范围内的操作将使用默认数据源,并且一旦beginSwitch,在end之前无法再次切换数据源,以此保证我们手工调用数据源切换后不会因为Mapper上存在注解而被AOP自动切换,我们在begin的时候传入数据源名称,并使用ThreadLocal保证多线程下的数据安全,另外我们重写了setTargetDataSources以保存当前设置的所有数据源,以便以后可以对其动态增加/移除
/**
* 动态数据源
* 该类内的方法线程安全
*/
public class MyDynamicDataSource extends AbstractRoutingDataSource {
private ThreadLocal currentDataSourceThreadLocal = new ThreadLocal<>();
private Map
动态数据源类建立好以后,我们就需要将他注入Spring的IoC中,我们需要从配置文件中读取数据源配置,所以需要定义好我们的数据源配置文件,这里写在application.yml中,贴代码
xxx:
data-sources:
- driverClassName: xxxx
url: xxxx
username: xxxx
password: xxxx
- driverClassName: xxxx
url: xxxx
username: xxxx
password: xxxx
建立映射该配置的配置类DataSourceConfigGroup
/**
* 数据源配置
*/
public class DataSourceConfigGroup {
private DruidDataSource[] dataSources;
public DruidDataSource[] getDataSources() {
return dataSources;
}
public void setDataSources(DruidDataSource[] dataSources) {
this.dataSources = dataSources;
}
}
既然完全从配置文件读取并且以此加载不同的bean,必定是要用spring-boot-starter的方式封装的,于是遵循其规范建立maven子项目my-db-spring-boot-starter,除了基本的mybatis,spring之类的包之外,还需要引入starter项目组件(starter项目就不深入了,有兴趣的自己去查查就行)到pom中
org.springframework.boot
spring-boot-configuration-processor
true
org.springframework.boot
spring-boot-autoconfigure
添加数据源自动配置类DataSourceAutoConfiguration
/**
* 默认数据源自动配置组件
*/
@ConditionalOnMissingBean(DataSource.class)
@ConditionalOnExpression("!'${xxx.data-sources[0].url:null}'.equalsIgnoreCase('null')") //存在数据源配置,则注入
public class DataSourceAutoConfiguration {
Logger logger = LoggerFactory.getLogger(DataSourceAutoConfiguration.class);
//注入datasource数据源配置
@Bean
@ConditionalOnMissingBean
@ConfigurationProperties("xxx") //注入数据源配置
public DataSourceConfigGroup dataSourceConfigGroup() {
return new DataSourceConfigGroup();
}
//注入DynamicDataSource动态多数据源
@Bean("defaultDataSource")
@ConditionalOnMissingBean(DataSource.class)
@ConditionalOnBean(DataSourceConfigGroup.class)
@ConditionalOnExpression("!'${xxx.data-sources[1].url:null}'.equalsIgnoreCase('null')") //如果存在第二个数据源,则为多数据源
public MyDynamicDataSource myDynamicDataSource(DataSourceConfigGroup config) {
DruidDataSource[] dataSources = config.getDataSources();
//多数据源环境
MyDynamicDataSource dynamicDataSource = new MyDynamicDataSource();
Map targetDataSource = new HashMap<>(dataSources.length);
for (DruidDataSource ds : dataSources) {
targetDataSource.put(ds.getName(), ds);
}
dynamicDataSource.setTargetDataSources(targetDataSource);
dynamicDataSource.setDefaultTargetDataSource(dataSources[0]); //设定第一个数据源为默认
return dynamicDataSource;
}
//注入单数据源
@Bean("defaultDataSource")
@ConditionalOnMissingBean({DataSource.class, MyDynamicDataSource.class})
@ConditionalOnBean(DataSourceConfigGroup.class)
public DataSource defaultDataSource(DataSourceConfigGroup config) {
return config.getDataSources()[0];
}
@Bean
@ConditionalOnBean(DataSource.class)
@ConditionalOnMissingBean(TransactionManagementConfigurer.class)
public TransactionManagementConfigurer transactionManagementConfigurer(DataSource dataSource) {
if (dataSource instanceof MyDynamicDataSource) {
logger.info("检测到数据库多数据源环境,但当前版本不支持XA事务,自动配置[默认Spring数据库事务管理器],请注意事务无法跨数据源");
} else {
logger.info("检测到数据库单数据源环境,自动配置[默认Spring数据库事务管理器]");
}
return () -> new DataSourceTransactionManager(dataSource);
}
}
该类会自动判断当前数据源配置,在只存在一个数据源节点时注入普通的DataSource,而在存在多个节点时自动注入我们自己建立的MyDynamicDataSource,这样就实现了动态数据源的自动装配.
为了使该配置生效,我们需要在该starter项目下的src/main/resource下建立META-INF/spring.factories文件,内容如下
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.xxx.autoconfig.DataSourceAutoConfiguration
为了使数据源自动切换,我们添加注解MyDataSource
/**
* 数据源指定注解
* 应用该注解的类和方法将默认使用指定的数据源来做数据库访问,但仍然可以使用MyDynamicDataSource.beginSwitch来切换数据源以覆盖该注解的数据源指定
*/
@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE,ElementType.METHOD})
public @interface MyDataSource {
/**
* 指定数据源名称
* @return
*/
String value();
}
然后建立AOP类DynamicDataSourceSwitchAspect切入所有具有该注解的类和方法来切换数据源,这样我们只需要在Mapper类或者其方法上加入该注解即可指定它默认使用的数据源
/**
* 动态数据源切换器
*/
@Aspect
public class DynamicDataSourceSwitchAspect implements Ordered {
private MyDynamicDataSource myDynamicDataSource;
public DynamicDataSourceSwitchAspect(MyDynamicDataSource myDynamicDataSource) {
this.myDynamicDataSource= myDynamicDataSource;
}
@Pointcut("@annotation(com.xxx.annotation.MyDataSource) || @within(com.xxx.annotation.MyDataSource)")
public void pointCut() {
}
@Around("pointCut()")
public Object execute(ProceedingJoinPoint pjp) throws Throwable {
// 代理对象
Method method = ((MethodSignature) pjp.getSignature()).getMethod();
//获取代理方法上的注解
MyDataSource anno = method.getAnnotation(MyDataSource.class);
if (anno == null) {
//获取代理类上的注解
anno = (MyDataSource) pjp.getSignature().getDeclaringType().getAnnotation(MyDataSource.class);
}
if (anno != null) {
boolean succ = false;
try {
succ = this.myDynamicDataSource.tryBeginSwitch(anno.value());
return pjp.proceed();
} finally {
if (succ) {
this.myDynamicDataSource.endSwitch();
}
}
} else {
return pjp.proceed();
}
}
@Override
public int getOrder() {
return 200;
}
}
注意使用try/finally包裹数据源的切换,并且判断在切换成功后才需要endSwitch,避免一些特殊情况可能造成的BUG,另外,由于使用了tryBeginSwitch,所以如果我们在代码中手动调用了数据源切换,则在该执行范围内,AOP就不会帮我们自动切换了
接下来就是将该AOP注入IoC容器了,注意,只有在多数据源模式下才需要用到该AOP,单数据源是不需要的,所以我们要加个Conditional判断是否存在动态数据源,代码如下
/**
* 动态数据源自动切换配置
*/
@AutoConfigureAfter(DataSourceAutoConfiguration.class)
@ConditionalOnBean(MyDynamicDataSource.class)
public class DynamicDataSourceAutoSwitchAutoConfiguration {
Logger logger = LoggerFactory.getLogger(DynamicDataSourceAutoSwitchAutoConfiguration.class);
@Bean
@ConditionalOnMissingBean
public DynamicDataSourceSwitchAspect dynamicDataSourceSwitchAspect(MyDynamicDataSource myDynamicDataSource) {
logger.info("检测到多数据源环境,自动配置[动态数据源自动切换AOP]");
return new DynamicDataSourceSwitchAspect(myDynamicDataSource);
}
}
然后修改META-INF/spring.factories文件,启用他的SpringBoot自动装配,内容如下
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.xxx.autoconfig.DataSourceAutoConfiguration,\
com.xxx.autoconfig.DynamicDataSourceAutoSwitchAutoConfiguration
springboot与mybatis的集成网上一大堆资料,但是由于使用了动态数据源,默认的Mybatis事务管理接口需要重写,另外由于本组件的mybatis接口所在包和xml所在路径都由配置文件指定,所以这部分并没有采用最常规的集成方案
首先我们的application.yml配置文件中对mybatis扫描的配置如下
xxx:
mybatis:
base-package: com.xxx
mapper-location: classpath*:com/xxx/dao/**/*.xml
建立映射类MyBatisConfig
/**
* MyBatis数据访问层基础包名
*/
public class MybatisConfig {
/**
* MyBatis接口所在的基础包
*/
private String[] basePackage = new String[]{"com.help.dao"};
/**
* MyBatis的XML映射文件所在位置
*/
private String[] mapperLocation = new String[]{"classpath*:com/help/dao/**/*.xml"};
public String[] getMapperLocation() {
return mapperLocation;
}
public void setMapperLocation(String[] mapperLocation) {
this.mapperLocation = mapperLocation;
}
public String[] getBasePackage() {
return basePackage;
}
public void setBasePackage(String[] basePackage) {
this.basePackage = basePackage;
}
}
由于MyBatis默认使用SpringManagedTransactionFactory作为事务管理器,而这个事务管理器在开启事务后会缓存当前connection,导致如果在多数据源环境下虽然切换了DataSource(不管是自动还是手动)但是仍然会使用原来的DataSource,我们要做的就是在多数据源环境下,如果检测到开启了事务但同时又切换数据源,则抛出错误提示不支持跨数据源的事务
建立匹配动态数据源的MyBatis事务工厂MyDynamicTransactionFactory
/**
* MyBatis动态数据源专用事务工厂
*/
public class MyDynamicTransactionFactory extends SpringManagedTransactionFactory {
@Override
public Transaction newTransaction(DataSource dataSource, TransactionIsolationLevel level, boolean autoCommit) {
if (dataSource instanceof MyDynamicDataSource) {
return new MyDynamicTransaction((MyDynamicDataSource) dataSource);
} else {
return super.newTransaction(dataSource, level, autoCommit);
}
}
}
建立与之匹配的事务类MyDynamicTransaction
/**
* MyBatis动态数据源专用事务
*/
public class MyDynamicTransaction implements Transaction {
private static final Logger logger = LoggerFactory.getLogger(MyDynamicTransaction.class);
private final MyDynamicDataSource myDynamicDataSource;
private DataSource dataSource;
private Connection connection;
private boolean isConnectionTransactional;
private boolean autoCommit;
public MyDynamicTransaction(MyDynamicDataSource dataSource) {
this.myDynamicDataSource= dataSource;
}
/**
* {@inheritDoc}
*/
@Override
public Connection getConnection() throws SQLException {
synchronized (this) {
if (this.dataSource == null) {
this.dataSource = myDynamicDataSource.getCurrentDataSource();
this.connection = openConnection(myDynamicDataSource);
return this.connection;
} else if (this.dataSource == myDynamicDataSource.getCurrentDataSource()) {
return this.connection;
} else {
throw new UnifyException("当前版本[HELP动态事务管理器]不支持跨数据源的Spring事务,请在同一个数据源下使用事务");
}
}
}
private Connection openConnection(MyDynamicDataSource dataSource) throws SQLException {
Connection connection = DataSourceUtils.getConnection(dataSource);
this.autoCommit = connection.getAutoCommit();
this.isConnectionTransactional = DataSourceUtils.isConnectionTransactional(connection, dataSource);
if (logger.isDebugEnabled()) {
if (isConnectionTransactional) {
logger.debug("已启用事务,数据库连接由[HELP动态事务管理器]管理 [" + connection + "]");
}
}
return connection;
}
/**
* {@inheritDoc}
*/
@Override
public void commit() throws SQLException {
if (this.connection != null && !this.isConnectionTransactional && !this.autoCommit) {
if (logger.isDebugEnabled()) {
logger.debug("数据库事务提交 [" + this.connection + "]");
}
this.connection.commit();
}
}
/**
* {@inheritDoc}
*/
@Override
public void rollback() throws SQLException {
if (this.connection != null && !this.isConnectionTransactional && !this.autoCommit) {
if (logger.isDebugEnabled()) {
logger.debug("数据库事务回滚 [" + this.connection + "]");
}
this.connection.rollback();
}
}
/**
* {@inheritDoc}
*/
@Override
public void close() throws SQLException {
DataSourceUtils.releaseConnection(this.connection, myDynamicDataSource);
}
/**
* {@inheritDoc}
*/
@Override
public Integer getTimeout() throws SQLException {
ConnectionHolder holder = (ConnectionHolder) TransactionSynchronizationManager.getResource(myDynamicDataSource);
if (holder != null && holder.hasTimeout()) {
return holder.getTimeToLiveInSeconds();
}
return null;
}
}
注意这里获得连接和关闭链接等操作必须使用动态数据源而不可以使用根据动态数据源获取到的实际数据源,不然获取到的连接将不具有事务性
建立MyBatis自动配置类MyBatisAutoConfiguration
/**
* MyBatis组件自动配置工具
*/
@ConditionalOnSingleCandidate(DataSource.class)
@ConditionalOnClass({MapperScannerConfigurer.class, SqlSessionFactory.class})
@AutoConfigureAfter({DataSourceAutoConfiguration.class})
public class MyBatisAutoConfiguration {
Logger logger = LoggerFactory.getLogger(MyBatisAutoConfiguration.class);
@Bean
@ConfigurationProperties("xxx.mybatis")
public MybatisConfig mybatisConfig() {
return new MybatisConfig();
}
//注入MyBatis的MapperScanner
@Bean
@ConditionalOnMissingBean
public MapperScannerConfigurer mapperScannerConfigurer(MybatisConfig mybatisConfig) {
MapperScannerConfigurer mapperScannerConfigurer = new MapperScannerConfigurer();
mapperScannerConfigurer.setBasePackage(StringUtil.join(mybatisConfig.getBasePackage(), ","));
mapperScannerConfigurer.setSqlSessionTemplateBeanName("defaultSqlSessionTemplate");
logger.info("检测到MyBatis环境,自动配置[MyBatis包扫描器],基础包目录:[" + StringUtil.join(helpMybatisConfig.getBasePackage(), ",") + "],xml文件所在路径[" + StringUtil.join(helpMybatisConfig.getMapperLocation(), ",") + "]");
return mapperScannerConfigurer;
}
//注入mybatis事务管理器,避免多数据源下的事务互串
@Bean
@ConditionalOnMissingBean
public TransactionFactory transactionFactory(DataSource dataSource) {
if (dataSource instanceof MyDynamicDataSource) {
logger.info("检测到多数据源环境,自动配置[自定义Mybatis动态事务管理器]");
return new MyDynamicTransactionFactory();
} else {
logger.info("检测到单数据源环境,自动配置[Spring-MyBatis默认事务管理器]");
return new SpringManagedTransactionFactory();
}
}
//注入mybatis的SqlSessionFactory
@Bean(name = "defaultSqlSessionFactory")
@ConditionalOnMissingBean(name = "defaultSqlSessionFactory")
public SqlSessionFactory sqlSessionFactory(DataSource dataSource, MybatisConfig mybatisConfig, @Autowired(required = false) List interceptors, @Autowired(required = false) TransactionFactory transactionFactory) {
org.apache.ibatis.session.Configuration conf = new org.apache.ibatis.session.Configuration();
conf.setMapUnderscoreToCamelCase(true);
conf.setLogImpl(Slf4jImpl.class);
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dataSource);
bean.setConfiguration(conf);
bean.setTransactionFactory(transactionFactory);
//添加插件
if (interceptors != null) {
bean.setPlugins(interceptors.toArray(new Interceptor[0]));
}
try {
String[] mapperLocation = mybatisConfig.getMapperLocation();
//添加XML目录
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
List resources = new ArrayList<>();
for (String s : mapperLocation) {
Resource[] all = resolver.getResources(s);
resources.addAll(Arrays.asList(all));
}
bean.setMapperLocations(resources.toArray(new Resource[resources.size()]));
logger.info("检测到MyBatis环境,自动配置[SqlSessionFactory]");
return bean.getObject();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
//注入MyBatis的SqlSessionTemplate
@Bean(name = "defaultSqlSessionTemplate")
@ConditionalOnMissingBean(name = "defaultSqlSessionTemplate")
public SqlSessionTemplate sqlSessionTemplate(@Qualifier("defaultSqlSessionFactory") SqlSessionFactory sqlSessionFactory) {
logger.info("检测到MyBatis环境,自动配置[SqlSessionTemplate]");
return new SqlSessionTemplate(sqlSessionFactory);
}
}
这里由于我的项目还集成了PageInterceptor分页插件和自己写的主键自动生成插件,所以注入了List
然后修改META-INF/spring.factories文件,启用他的SpringBoot自动装配,内容如下
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.xxx.autoconfig.DataSourceAutoConfiguration,\
com.xxx.autoconfig.DynamicDataSourceAutoSwitchAutoConfiguration,\
com.xxx.autoconfig.MyBatisAutoConfiguration
建立类DruidFilterAutoConfiguration
@ConditionalOnClass({HttpServlet.class, Filter.class})
@AutoConfigureAfter(DataSourceAutoConfiguration.class)
@ConditionalOnBean({DataSourceAutoConfiguration.class})
@ConditionalOnWebApplication
public class DruidFilterAutoConfiguration {
Logger logger = LoggerFactory.getLogger(DruidFilterAutoConfiguration.class);
@Bean
@ConditionalOnMissingBean(value = WebStatFilter.class, parameterizedContainer = FilterRegistrationBean.class)
public FilterRegistrationBean druidStatFilterRegister() {
FilterRegistrationBean registration = new FilterRegistrationBean();
registration.setFilter(new WebStatFilter());
registration.addUrlPatterns("/*");
registration.setName("druidWebStatFilter");
registration.setOrder(5);
registration.addInitParameter("exclusions", "*.js,*.gif,*.jpg,*.bmp,*.png,*.css,*.ico,/druid/*");
logger.info("检测到WEB环境,自动配置[Druid数据采集器]");
return registration;
}
@Bean
@ConfigurationProperties("druid")
public DruidFilterConfig helpDruidFilterConfig() {
return new DruidFilterConfig();
}
@Bean
@ConditionalOnBean({DruidFilterConfig.class, DataSource.class})
@ConditionalOnMissingBean(value = StatViewServlet.class, parameterizedContainer = ServletRegistrationBean.class)
public ServletRegistrationBean druidStatViewServletRegister(DruidFilterConfig druidFilterConfig, @Autowired List dataSources) {
for (DataSource ds : dataSources) {
if (ds instanceof DruidDataSource) {
try {
if (((DruidDataSource) ds).getFilterClassNames() == null || ((DruidDataSource) ds).getFilterClassNames().size() == 0) {
((DruidDataSource) ds).setFilters("stat");
}
} catch (SQLException e) {
logger.warn("为Druid数据源注入数据库监控失败[" + e.getMessage() + "]", e);
}
} else if (ds instanceof MyDynamicDataSource) {
Map map = ((MyDynamicDataSource) ds).getTargetDataSources();
if (map != null && map.size() > 0) {
Collection targets = map.values();
for (Object o : targets) {
if (o instanceof DruidDataSource) {
try {
((DruidDataSource) o).setFilters("stat");
} catch (SQLException e) {
logger.warn("为Druid数据源[" + ((DruidDataSource) o).getName() + "]注入数据库监控失败[" + e.getMessage() + "]", e);
}
}
}
}
}
}
ServletRegistrationBean bean = new ServletRegistrationBean();
bean.addUrlMappings("/druid/*");
bean.addInitParameter("loginUsername", druidFilterConfig.getLoginUsername()); //用户名
bean.addInitParameter("loginPassword", druidFilterConfig.getLoginPassword()); // 密码
bean.addInitParameter("resetEnable", "false"); // 禁用HTML页面上的“Reset All”功能
bean.setServlet(new StatViewServlet());
//bean.addInitParameter("allow",""); // IP白名单 (没有配置或者为空,则允许所有访问)
//bean.addInitParameter("deny",""); // IP黑名单 (存在共同时,deny优先于allow)
logger.info("检测到WEB环境,自动配置[Druid监控界面],访问路径[/druid]");
return bean;
}
/**
* Druid监控过滤器配置
*/
public class DruidFilterConfig {
private String loginUsername = "admin";
private String loginPassword = "123456";
public String getLoginUsername() {
return loginUsername;
}
public void setLoginUsername(String loginUsername) {
this.loginUsername = loginUsername;
}
public String getLoginPassword() {
return loginPassword;
}
public void setLoginPassword(String loginPassword) {
this.loginPassword = loginPassword;
}
}
}
主要行为是检测当前为Web环境则添加Druid监控Servlet,并根据当前的DataSource类型动态为其添加过滤器
经过上面的4步,项目的多数据源集成就已经全部完成了,那么如果我们想要在项目运行过程中动态添加数据源,代码可以按下面的方法写
@Autowired(required = false)
MyDynamicDataSource myDynamicDataSource;
@GetMapping("/test")
public String datasource() throws SQLException {
String newName = "ds"; //数据源名称
Map target = myDynamicDataSource.getTargetDataSources();
if (!target.containsKey(newName)) {
DruidDataSource ds = new DruidDataSource();
ds.setName(newName);
ds.setUrl("jdbc:mysql://xxxxxxxxxxx");
ds.setDriverClassName("com.mysql.jdbc.Driver");
ds.setUsername("xxxx");
ds.setPassword("xxxx");
ds.setFilters("stat");
target.put(newName, ds);
myDynamicDataSource.afterPropertiesSet();
}
return newName;
}
手动切换数据源的代码如下(以PParamMapper为例)
@Autowired
PParamMapper pParamMapper;
@Autowired(required = false)
MyDynamicDataSource myDynamicDataSource;
@GetMapping(value = "/test2")
public String test() {
PParam pParam = new PParam();
pParam.setParamKey("AAA");
pParam.setParamValue("AAA");
pParamMapper.insert(pParam);
boolean succ = false;
try {
succ = myDynamicDataSource.tryBeginSwitch("ds");
pParamMapper.insert(pParam);
}finally {
if(succ){
myDynamicDataSource.endSwitch();
}
}
return "SUCCESS";
}
这段代码会先将AAA数据插入PParamMapper注解上指定的数据源(如果没有注解则插入配置文件中配置的第一个数据源),然后再将AAA插入名为ds的数据源
全文完