1. 背景
我们一般应用对数据库而言都是“读多写少”,也就说对数据库读取数据的压力比较大,有一个思路就是说采用数据库集群的方案,
其中一个是主库,负责写入数据,我们称之为:写库;
其它都是从库,负责读取数据,我们称之为:读库;
那么,对我们的要求是:
1、 读库和写库的数据一致;
2、 写数据必须写到写库;
3、 读数据必须到读库;
2. 方案
解决读写分离的方案有两种:应用层解决和中间件解决。
2.1. 应用层解决:
优点:
1、多数据源切换方便,由程序自动完成;
2、不需要引入中间件;
3、理论上支持任何数据库;
缺点:
1、由程序员完成,运维参与不到;
2、不能做到动态增加数据源;
2.2. 中间件解决
优缺点:
优点:
1、源程序不需要做任何改动就可以实现读写分离;
2、动态添加数据源不需要重启程序;
缺点:
1、程序依赖于中间件,会导致切换数据库变得困难;
2、由中间件做了中转代理,性能有所下降;
相关中间件产品使用:
mysql-proxy:http://hi.baidu.com/geshuai2008/item/0ded5389c685645f850fab07
Amoeba for MySQL:http://www.iteye.com/topic/188598和http://www.iteye.com/topic/1113437
3. 使用Spring基于应用层实现
3.1. 原理
在进入Service之前,使用AOP来做出判断,是使用写库还是读库,判断依据可以根据方法名判断,比如说以query、find、get等开头的就走读库,其他的走写库。
3.2. DynamicDataSource
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
public class DynamicDataSource extends AbstractRoutingDataSource{
@Override
protected Object determineCurrentLookupKey() {
return DynamicDataSourceHolder.getDataSourceKey();
}
}
3.3. DynamicDataSourceHolder
"code" class = "java" >
public class DynamicDataSourceHolder {
private static final String MASTER = "master" ;
private static final String SLAVE = "slave" ;
private static final ThreadLocal holder = new ThreadLocal();
public static void putDataSourceKey(String key) {
holder.set(key);
}
public static String getDataSourceKey() {
return holder.get();
}
public static void markMaster(){
putDataSourceKey(MASTER);
}
public static void markSlave(){
putDataSourceKey(SLAVE);
}
}
3.4. DataSourceAspect
import org.apache.commons.lang3.StringUtils;
import org.aspectj.lang.JoinPoint;
public class DataSourceAspect {
public void before(JoinPoint point) {
String methodName = point.getSignature().getName();
if (isSlave(methodName)) {
DynamicDataSourceHolder.markSlave();
} else {
DynamicDataSourceHolder.markMaster();
}
}
private Boolean isSlave(String methodName) {
return StringUtils.startsWithAny(methodName, "query" , "find" , "get" );
}
}
3.5. 配置2个数据源
3.5.1. jdbc.properties
jdbc.master.driver=com.mysql.jdbc.Driver
jdbc.master.url=jdbc:mysql:
jdbc.master.username=root
jdbc.master.password=123456
jdbc.slave01.driver=com.mysql.jdbc.Driver
jdbc.slave01.url=jdbc:mysql:
jdbc.slave01.username=root
jdbc.slave01.password=123456
3.5.2. 定义连接池
< bean id = "masterDataSource" class = "com.jolbox.bonecp.BoneCPDataSource"
destroy-method = "close" >
< property name = "driverClass" value = "${jdbc.master.driver}" />
< property name = "jdbcUrl" value = "${jdbc.master.url}" />
< property name = "username" value = "${jdbc.master.username}" />
< property name = "password" value = "${jdbc.master.password}" />
< property name = "idleConnectionTestPeriod" value = "60" />
< property name = "idleMaxAge" value = "30" />
< property name = "maxConnectionsPerPartition" value = "150" />
< property name = "minConnectionsPerPartition" value = "5" />
bean >
< bean id = "slave01DataSource" class = "com.jolbox.bonecp.BoneCPDataSource"
destroy-method = "close" >
< property name = "driverClass" value = "${jdbc.slave01.driver}" />
< property name = "jdbcUrl" value = "${jdbc.slave01.url}" />
< property name = "username" value = "${jdbc.slave01.username}" />
< property name = "password" value = "${jdbc.slave01.password}" />
< property name = "idleConnectionTestPeriod" value = "60" />
< property name = "idleMaxAge" value = "30" />
< property name = "maxConnectionsPerPartition" value = "150" />
< property name = "minConnectionsPerPartition" value = "5" />
bean >
3.5.3. 定义DataSource
< bean id = "dataSource" class = "cn.itcast.usermanage.spring.DynamicDataSource" >
< property name = "targetDataSources" >
< map key-type = "java.lang.String" >
< entry key = "master" value-ref = "masterDataSource" />
< entry key = "slave" value-ref = "slave01DataSource" />
map >
property >
< property name = "defaultTargetDataSource" ref = "masterDataSource" />
bean >
3.6. 配置事务管理以及动态切换数据源切面
3.6.1. 定义事务管理器
< bean id = "transactionManager"
class = "org.springframework.jdbc.datasource.DataSourceTransactionManager" >
< property name = "dataSource" ref = "dataSource" />
bean >
3.6.2. 定义事务策略
< tx:advice id = "txAdvice" transaction-manager = "transactionManager" >
< tx:attributes >
< tx:method name = "query*" read-only = "true" />
< tx:method name = "find*" read-only = "true" />
< tx:method name = "get*" read-only = "true" />
< tx:method name = "save*" propagation = "REQUIRED" />
< tx:method name = "update*" propagation = "REQUIRED" />
< tx:method name = "delete*" propagation = "REQUIRED" />
< tx:method name = "*" />
tx:attributes >
tx:advice >
3.6.3. 定义切面
< bean class = "cn.itcast.usermanage.spring.DataSourceAspect" id = "dataSourceAspect" />
< aop:config >
< aop:pointcut id = "txPointcut" expression = "execution(* xx.xxx.xxxxxxx.service.*.*(..))" />
< aop:advisor advice-ref = "txAdvice" pointcut-ref = "txPointcut" />
< aop:aspect ref = "dataSourceAspect" order = "-9999" >
< aop:before method = "before" pointcut-ref = "txPointcut" />
aop:aspect >
aop:config >
4. 改进切面实现,使用事务策略规则匹配
之前的实现我们是将通过方法名匹配,而不是使用事务策略中的定义,我们使用事务管理策略中的规则匹配。
4.1. 改进后的配置
< span style = "white-space:pre" > span >
< bean class = "cn.itcast.usermanage.spring.DataSourceAspect" id = "dataSourceAspect" >
< property name = "txAdvice" ref = "txAdvice" />
< property name = "slaveMethodStart" value = "query,find,get" />
bean >
4.2. 改进后的实现
5. 一主多从的实现
很多实际使用场景下都是采用“一主多从”的架构的,所有我们现在对这种架构做支持,目前只需要修改DynamicDataSource即可。
5.1. 实现
import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import javax.sql.DataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
import org.springframework.util.ReflectionUtils;
public class DynamicDataSource extends AbstractRoutingDataSource {
private static final Logger LOGGER = LoggerFactory.getLogger(DynamicDataSource. class );
private Integer slaveCount;
private AtomicInteger counter = new AtomicInteger(- 1 );
private List slaveDataSources = new ArrayList( 0 );
@Override
protected Object determineCurrentLookupKey() {
if (DynamicDataSourceHolder.isMaster()) {
Object key = DynamicDataSourceHolder.getDataSourceKey();
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("当前DataSource的key为: " + key);
}
return key;
}
Object key = getSlaveKey();
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("当前DataSource的key为: " + key);
}
return key;
}
@SuppressWarnings ( "unchecked" )
@Override
public void afterPropertiesSet() {
super .afterPropertiesSet();
Field field = ReflectionUtils.findField(AbstractRoutingDataSource.class , "resolvedDataSources" );
field.setAccessible(true );
try {
Map resolvedDataSources = (Map) field.get(this );
this .slaveCount = resolvedDataSources.size() - 1 ;
for (Map.Entry entry : resolvedDataSources.entrySet()) {
if (DynamicDataSourceHolder.MASTER.equals(entry.getKey())) {
continue ;
}
slaveDataSources.add(entry.getKey());
}
} catch (Exception e) {
LOGGER.error("afterPropertiesSet error! " , e);
}
}
public Object getSlaveKey() {
Integer index = counter.incrementAndGet() % slaveCount;
if (counter.get() > 9999 ) {
counter.set(-1 );
}
return slaveDataSources.get(index);
}
}
6. MySQL主从复制
6.1. 原理
mysql主(称master)从(称slave)复制的原理:
1、master将数据改变记录到二进制日志(binarylog)中,也即是配置文件log-bin指定的文件(这些记录叫做二进制日志事件,binary log events)
2、slave将master的binary logevents拷贝到它的中继日志(relay log)
3、slave重做中继日志中的事件,将改变反映它自己的数据(数据重演)
6.2. 主从配置需要注意的地方
1、主DB server和从DB server数据库的版本一致
2、主DB server和从DB server数据库数据一致[ 这里就会可以把主的备份在从上还原,也可以直接将主的数据目录拷贝到从的相应数据目录]
3、主DB server开启二进制日志,主DB server和从DB server的server_id都必须唯一
6.3. 主库配置(windows,Linux下也类似)
在my.ini修改:
#开启主从复制,主库的配置
log-bin= mysql3306-bin
#指定主库serverid
server-id=101
#指定同步的数据库,如果不指定则同步全部数据库
binlog-do-db=mybatis_1128
执行SQL语句查询状态: SHOW MASTER STATUS
需要记录下 Position 值,需要在从库中设置同步起始值。
6.4. 在主库创建同步用户
#授权用户slave01使用123456密码登录mysql
grant replication slave on *.* to 'slave01'@'127.0.0.1'identified by '123456';
flush privileges;
6.5. 从库配置
在my.ini修改:
#指定serverid,只要不重复即可,从库也只有这一个配置,其他都在SQL语句中操作
server-id=102
以下执行SQL:
CHANGEMASTER TO
master_host='127.0.0.1',
master_user='slave01',
master_password='123456',
master_port=3306,
master_log_file='mysql3306-bin.000006',
master_log_pos=1120;
#启动slave同步
STARTSLAVE;
#查看同步状态
SHOWSLAVE STATUS;
7. 参考资料
http://www.iteye.com/topic/1127642
http://634871.blog.51cto.com/624871/1329301