springboot+JPA+druid多数据源配置
title: springboot+JPA+druid多数据源配置
copyright: true
categories: springmvc和springboot
tags: [springboot,JPA,多数据源配置,druid]
password:
上次发了一篇mysql主从复制的博客，就我正在做的项目来讲，项目开始之初就是两个数据库服务器，一个master，一个slave，运维已经做好了主从复制的架构，但是基本上用的都是master数据库，从库基本都没怎么用。正好有一个关于统计的需求查询效率有点低，也比较频繁，就想着还不如查从库了。然后就去研究了一下springboot+JPA环境下的多数据源和读写分离。
文章目录
- 一、首先要配置druid数据源
- 二、数据源生成类：
- 三、EntityManager生成类
- 四、问题汇总
一、首先要配置druid数据源
1、druid的某个配置很重要，因为数据库的每个连接是默认有一个8小时的超时时间的，如果过了8小时，连接会自动断掉，而druid此时还不知道连接已经断掉，也没有回收，一直用那个连接就会报错。此种情况一般出现于定时任务，比如某个统计需要每天统计一次，某个连接可能24小时才会使用一次，这时就会出现这个问题。
解决方案：
①、更改数据库默认的wait_outtime，改长一点，不推荐
②、可以从数据源入手，更改druid的配置，让druid每隔一段时间自动去检测连接池中每个连接是否可用，如果不可用则回收。后面需要连接时，会重新开启一个连接。这个时间间隔需要小于数据库的超时时间才对。
2、druid配置和jpa配置
①、主库配置：
# JDBC配置
spring.datasource.druid.url= jdbc:mysql://10.10.10.105:23306/xiuba_user?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=round&autoReconnect=true&useSSL=false
spring.datasource.druid.username= xiuba
spring.datasource.druid.driver-class-name= com.mysql.jdbc.Driver
spring.datasource.druid.password= xiuba123456
# 连接池配置，说明请参考Druid Wiki，DruidDataSource配置属性列表
spring.datasource.druid.initial-size=3
spring.datasource.druid.max-active=20
spring.datasource.druid.min-idle=3
spring.datasource.druid.max-wait=60000
spring.datasource.druid.time-between-eviction-runs-millis=50000
spring.datasource.druid.min-evictable-idle-time-millis=80000
spring.datasource.druid.max-evictable-idle-time-millis=100000
spring.datasource.druid.validation-query=select 'x'
spring.datasource.druid.validation-query-timeout=10
spring.datasource.druid.test-on-borrow=false
spring.datasource.druid.test-on-return=false
spring.datasource.druid.test-while-idle=true
#默认值stat，配置多个英文逗号分隔
spring.datasource.druid.filters= stat,wall,slf4j
# WebStatFilter配置，说明请参考Druid Wiki，配置_配置WebStatFilter
spring.datasource.druid.web-stat-filter.enabled=true
spring.datasource.druid.web-stat-filter.urlPattern=/*
spring.datasource.druid.web-stat-filter.exclusions=*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*
spring.datasource.druid.web-stat-filter.sessionStatMaxCount=600
spring.datasource.druid.web-stat-filter.sessionStatEnable=true
#spring.datasource.druid.web-stat-filter.principalSessionName=
#spring.datasource.druid.web-stat-filter.principalCookieName=
spring.datasource.druid.web-stat-filter.profileEnable=true
# StatViewServlet配置，说明请参考Druid Wiki，配置_StatViewServlet配置
spring.datasource.druid.stat-view-servlet.enabled=true
spring.datasource.druid.stat-view-servlet.urlPattern=/druid/*
spring.datasource.druid.stat-view-servlet.resetEnable=true
spring.datasource.druid.stat-view-servlet.loginUsername=druid
spring.datasource.druid.stat-view-servlet.loginPassword=123456
#spring.datasource.druid.stat-view-servlet.allow=
#spring.datasource.druid.stat-view-servlet.deny=
②、从库配置
#从库配置
# JDBC配置
spring.datasource.druid.slave.url= jdbc:mysql://10.10.10.104:23306/xiuba_user?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=round&autoReconnect=true&useSSL=false
spring.datasource.druid.slave.username=xiuba_select
spring.datasource.druid.slave.driver-class-name= com.mysql.jdbc.Driver
spring.datasource.druid.slave.password=xiuba123456
# 连接池配置，说明请参考Druid Wiki，DruidDataSource配置属性列表
spring.datasource.druid.slave.initial-size=3
spring.datasource.druid.slave.max-active=20
spring.datasource.druid.slave.min-idle=3
spring.datasource.druid.slave.max-wait=60000
spring.datasource.druid.slave.time-between-eviction-runs-millis=50000
spring.datasource.druid.slave.min-evictable-idle-time-millis=80000
spring.datasource.druid.slave.max-evictable-idle-time-millis=100000
spring.datasource.druid.slave.validation-query=select 'x'
spring.datasource.druid.slave.validation-query-timeout=10
spring.datasource.druid.slave.test-on-borrow=false
spring.datasource.druid.slave.test-on-return=false
spring.datasource.druid.slave.test-while-idle=true
#默认值stat，配置多个英文逗号分隔
spring.datasource.druid.slave.filters= stat,wall,slf4j
③、JPA配置
spring.jpa.properties.hibernate.hbm2ddl.auto= update
spring.jpa.open-in-view=true
spring.jpa.properties.hibernate.show_sql=false
spring.jpa.properties.hibernate.format_sql=false
注意：
①： 因为jpa或者说hibernate是orm框架，有一个自动更新数据库的配置，可以使数据库根据项目中的实体类自动生成相应的表结构，非常方便好用。但是这也引来了一些问题，如果从库也应用这个配置，就会更改从库的表结构，而从库只要自己发生变化而不是主库复制来的，就会发生mysql错误，使主从复制停止，造成重大错误。所以此时jpa的配置应该主从分离，此操作实现在EntityManager生成配置类中。
②：为了避免上述情况在我们不经意间发生，最好给从库配置一个只能读不能写的权限的用户，我们所有操作都用这个用户来执行，避免人为错误。
二、数据源生成类：
package com.zhibi.xiuba.config.data;
import com.alibaba.druid.spring.boot.autoconfigure.DruidDataSourceBuilder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.env.Environment;
import javax.sql.DataSource;
/**
* Created by QinHe on 2018-07-24.
*/
@Configuration
public class DruidDataSourceConfig {
/**
* 主DataSource 配置
*/
@Primary
@ConfigurationProperties(prefix = "spring.datasource.druid")
@Bean(name = "primaryDruidDataSource")
public DataSource primaryDruidDataSource(Environment environment) {
return DruidDataSourceBuilder.create().build(environment, "spring.datasource.druid.");
}
/**
* 从DataSource 配置
*/
@ConfigurationProperties(prefix = "spring.datasource.druid.slave")
@Bean(name = "slaveDruidDataSource")
public DataSource slaveDruidDataSource(Environment environment) {
return DruidDataSourceBuilder.create().build(environment, "spring.datasource.druid.slave.");
}
}
三、EntityManager生成类
与jpa结合起来：
1、主库：
package com.zhibi.xiuba.config.data;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.orm.jpa.JpaProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.annotation.Resource;
import javax.persistence.EntityManager;
import javax.sql.DataSource;
import java.util.Map;
/**
* Created by QinHe on 2018-07-24.
*/
@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(
entityManagerFactoryRef = "entityManagerFactoryPrimary",
transactionManagerRef = "transactionManagerPrimary",
basePackages = {"com.zhibi.xiuba.repository.primary"}) //设置Repository所在位置
public class PrimaryConfig {
@Resource(name = "primaryDruidDataSource")
private DataSource primaryDruidDataSource;
@Autowired
private JpaProperties jpaProperties;
@Primary
@Bean(name = "entityManagerFactoryPrimary")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryPrimary(EntityManagerFactoryBuilder builder) {
return builder
.dataSource(primaryDruidDataSource)
.properties(getVendorProperties(primaryDruidDataSource))
.packages("com.zhibi.xiuba") //设置实体类所在位置
.persistenceUnit("primaryPersistenceUnit")
.build();
}
@Primary
@Bean(name = "entityManagerPrimary")
public EntityManager entityManagerPrimary(EntityManagerFactoryBuilder builder) {
return entityManagerFactoryPrimary(builder).getObject().createEntityManager();
}
private Map getVendorProperties(DataSource dataSource) {
return jpaProperties.getHibernateProperties(dataSource);
}
@Primary
@Bean(name = "transactionManagerPrimary")
public PlatformTransactionManager transactionManagerPrimary(EntityManagerFactoryBuilder builder) {
return new JpaTransactionManager(entityManagerFactoryPrimary(builder).getObject());
}
}
2、从库
package com.zhibi.xiuba.config.data;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.orm.jpa.JpaProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.annotation.Resource;
import javax.persistence.EntityManager;
import javax.sql.DataSource;
import java.util.Map;
/**
* Created by QinHe on 2018-07-24.
*/
@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(
entityManagerFactoryRef = "entityManagerFactorySlave",
transactionManagerRef = "transactionManagerSlave",
basePackages = {"com.zhibi.xiuba.repository.slave"}) //设置Repository所在位置
public class SlaveConfig {
@Resource(name = "slaveDruidDataSource")
private DataSource slaveDruidDataSource;
@Autowired
private JpaProperties jpaProperties;
@Bean(name = "entityManagerFactorySlave")
public LocalContainerEntityManagerFactoryBean entityManagerFactorySlave(EntityManagerFactoryBuilder builder) {
LocalContainerEntityManagerFactoryBean slavePersistenceUnit = builder
.dataSource(slaveDruidDataSource)
.properties(getVendorProperties(slaveDruidDataSource))
.packages("com.zhibi.xiuba") //设置实体类所在位置
.persistenceUnit("slavePersistenceUnit")
.build();
slavePersistenceUnit.getJpaPropertyMap().remove("hibernate.hbm2ddl.auto");
return slavePersistenceUnit;
}
@Bean(name = "entityManagerSlave")
public EntityManager entityManagerSlave(EntityManagerFactoryBuilder builder) {
return entityManagerFactorySlave(builder).getObject().createEntityManager();
}
private Map getVendorProperties(DataSource dataSource) {
return jpaProperties.getHibernateProperties(dataSource);
}
@Bean(name = "transactionManagerSlave")
public PlatformTransactionManager transactionManagerSlave(EntityManagerFactoryBuilder builder) {
return new JpaTransactionManager(entityManagerFactorySlave(builder).getObject());
}
}
四、问题汇总
1、在service层引入entityManager的时候，用spring的@Autowired或者@Resource引入会有一些问题，有时候数据源中的连接的获取会不正常，所以用官方的方法引入比较好，比如引入从库的entityManager：
@PersistenceContext(unitName = "entityManagerFactorySlave")
private EntityManager entityManagerSlave;
2、如果两个实体之间具有管理关系，并且是懒加载的，在从库查询时会有no session问题，就算配置了spring.jpa.open-in-view=true，还是会有问题。springboot对于这个配置是这样处理的：
JpaBaseConfiguration.java源码：
@Configuration
@ConditionalOnWebApplication
@ConditionalOnClass(WebMvcConfigurerAdapter.class)
@ConditionalOnMissingBean({ OpenEntityManagerInViewInterceptor.class,
OpenEntityManagerInViewFilter.class })
@ConditionalOnProperty(prefix = "spring.jpa", name = "open-in-view", havingValue = "true", matchIfMissing = true)
protected static class JpaWebConfiguration {
// Defined as a nested config to ensure WebMvcConfigurerAdapter is not read when
// not on the classpath
@Configuration
protected static class JpaWebMvcConfiguration extends WebMvcConfigurerAdapter {
@Bean
public OpenEntityManagerInViewInterceptor openEntityManagerInViewInterceptor() {
return new OpenEntityManagerInViewInterceptor();
}
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addWebRequestInterceptor(openEntityManagerInViewInterceptor());
}
}
}
jpa底层判断spring.jpa.open-in-view是否为true，如果是true，则注册OpenEntityManagerInViewInterceptor这个拦截器。
OpenEntityManagerInViewInterceptor对于懒加载的处理：
@Override
public void preHandle(WebRequest request) throws DataAccessException {
String participateAttributeName = getParticipateAttributeName();
WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request);
if (asyncManager.hasConcurrentResult()) {
if (applyCallableInterceptor(asyncManager, participateAttributeName)) {
return;
}
}
if (TransactionSynchronizationManager.hasResource(getEntityManagerFactory())) {
// Do not modify the EntityManager: just mark the request accordingly.
Integer count = (Integer) request.getAttribute(participateAttributeName, WebRequest.SCOPE_REQUEST);
int newCount = (count != null ? count + 1 : 1);
request.setAttribute(getParticipateAttributeName(), newCount, WebRequest.SCOPE_REQUEST);
}
else {
logger.debug("Opening JPA EntityManager in OpenEntityManagerInViewInterceptor");
try {
EntityManager em = createEntityManager();
EntityManagerHolder emHolder = new EntityManagerHolder(em);
TransactionSynchronizationManager.bindResource(getEntityManagerFactory(), emHolder);
AsyncRequestInterceptor interceptor = new AsyncRequestInterceptor(getEntityManagerFactory(), emHolder);
asyncManager.registerCallableInterceptor(participateAttributeName, interceptor);
asyncManager.registerDeferredResultInterceptor(participateAttributeName, interceptor);
}
catch (PersistenceException ex) {
throw new DataAccessResourceFailureException("Could not create JPA EntityManager", ex);
}
}
}
@Override
public void postHandle(WebRequest request, ModelMap model) {
}
@Override
public void afterCompletion(WebRequest request, Exception ex) throws DataAccessException {
if (!decrementParticipateCount(request)) {
EntityManagerHolder emHolder = (EntityManagerHolder)
TransactionSynchronizationManager.unbindResource(getEntityManagerFactory());
logger.debug("Closing JPA EntityManager in OpenEntityManagerInViewInterceptor");
EntityManagerFactoryUtils.closeEntityManager(emHolder.getEntityManager());
}
}
在请求到服务端时，先经过这个拦截器，将getEntityManagerFactory()作为key，将EntityManager em = createEntityManager();EntityManagerHolder emHolder = new EntityManagerHolder(em);emHolder作为值，存入TransactionSynchronizationManager的当前线程的ThreadLocal变量中，在后面的懒加载的处理，就可以通过entitymanager去加载数据，但是这个拦截器getEntityManagerFactory()和createEntityManager()都是用的默认的entityManager（主库的），所以对于从库的查询就不行了，为了解决这个问题，需要在方法中显式的存入entityManager，如下：
/**
* 条件查询
*/
@Override
// @Transactional(value = "transactionManagerSlave", rollbackFor = Exception.class)
public Page<User> getUserListByCondition(Params params, Pageable pageable, String platForm) {
EntityManager entityManagerSlave = null;
try {
if (!TransactionSynchronizationManager.hasResource(
entityManagerFactorySlave)) {
entityManagerSlave = entityManagerFactorySlave.createEntityManager();
EntityManagerHolder entityManagerHolder = new EntityManagerHolder(entityManagerSlave);
// TransactionSynchronizationManager.unbindResource(entityManagerFactorySlave);
TransactionSynchronizationManager.bindResource(entityManagerFactorySlave, entityManagerHolder);
log.info("userService:getUserListByCondition线程:" + Thread.currentThread().getName() + "绑定事务！");
} else {
log.info("userService:getUserListByCondition线程:" + Thread.currentThread().getName() + "已经有事务了！");
}
Page<User> all = userSlaveRepository.findAll(getUserSearchWhereClause(params, platForm), pageable);
List<User> list = all.getContent();
try {
changeUserListFor(list);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
return null;
}
return all;
} catch (IllegalStateException e) {
e.printStackTrace();
return null;
} finally {
if (TransactionSynchronizationManager.hasResource(entityManagerFactorySlave)) {
TransactionSynchronizationManager.unbindResource(entityManagerFactorySlave);
}
if (entityManagerSlave != null) {
EntityManagerFactoryUtils.closeEntityManager(entityManagerSlave);
}
}
}