分布式或者springboot+mybatis集成sharding实现分库分表的代码实现详细过程。
前言
众所周知所谓的分布式或者spring cloud都是基于springboot的业务横向扩展，在业务处理上面做了拆分。因此在本质上面并没有大的变化。因此在集成方面不管是springboot还是springcloud都可以集成一些中间件。这是我个人对于学习springcloud的时候的突然感悟，因为之前我一直认为springcloud与springboot不同，但是后来觉得其实都是差不多的东西，只是使用的业务场景不一样，选择的框架不一样罢了。
实现前提：
- 对maven依赖有一定的熟练度。
- 能够独立搭建spring cloud项目或者完整的spring boot项目并且集成mybatis对数据库的访问，使用mybatis的逆向工程完成部分代码的生成（方便使用）
- 能够理解配置类的加载顺序和加载时间。
- 对mybatis，sharding有一定的理解。（可查看学习官方文档）
相关依赖：
提示：除了一些常见的依赖还需要导入sharding的依赖
io.shardingsphere
sharding-jdbc-core
3.1.0
需要参考的application.yml文件
spring:
datasource-master:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/test1?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-slave:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/test1?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-order:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_order?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-seat-1:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_seat_1?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-seat-2:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_seat_2?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-seat-3:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_seat_3?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-seat-4:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_seat_4?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root
datasource-seat-5:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://localhost:3306/t_train_seat_5?characterEncoding=UTF-8&useSSL=false&useUnicode=true&connectTimeout=1000&socketTimeout=60000&allowMultiQueries=true
username: root
password: root说明：这个配置文件里面配置的有主从数据库和其他的分库数据库的相关信息。
实现原理：
分库分表是根据某一个具有代表性的字段进行拆分，比如说有一个表空间（schema）student_database_1,student_database_2他们公共的字段是student_database_因此在访问的时候拼接后面的字段就行了。分库也是同样的原理，比如根据年份进行分表：table_2021,table_2022只要判断传入的对象中的这个字段的年份信息就能够自动进行分表进行查询。因此在这之前一定要创建好相关的表结构，表空间结构。
准备工作完成后开始代码实现的阶段：
数据库主从配置：
config配置类
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import io.shardingsphere.api.algorithm.masterslave.RoundRobinMasterSlaveLoadBalanceAlgorithm;
import io.shardingsphere.api.config.rule.MasterSlaveRuleConfiguration;
import io.shardingsphere.shardingjdbc.api.MasterSlaveDataSourceFactory;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import javax.sql.DataSource;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
/**
* @ClassName: BasicDataSourceConfig
* @Description: 主从数据源配置
* @Author: TXW
* @Date: 2022/7/15
*/
@Configuration
@MapperScan(basePackages = "com.next.mapper",sqlSessionTemplateRef = "sqlSessionTemplate")
public class BasicDataSourceConfig {
@Primary
@Bean(name = DataSources.MASTER_DB)
@ConfigurationProperties(prefix = "spring.datasource-master")
public DataSource masterDB(){
return DataSourceBuilder.create().build();
}
@Bean(name = DataSources.SLAVE_DB)
@ConfigurationProperties(prefix = "spring.datasource-slave")
public DataSource slaveDB(){
return DataSourceBuilder.create().build();
}
@Bean(name = "masterSlaveDatasource") //主从数据源 使用插件shardingsphere
public DataSource masterSlaveDatasource(@Qualifier(DataSources.MASTER_DB) DataSource masterDb,
@Qualifier(DataSources.SLAVE_DB) DataSource slaveDb) throws SQLException {
Map map = new HashMap<>();
map.put(DataSources.MASTER_DB,masterDb);
map.put(DataSources.SLAVE_DB,slaveDb);
MasterSlaveRuleConfiguration masterSlaveRuleConfiguration = new MasterSlaveRuleConfiguration(
"db_master_slave", //名称
DataSources.MASTER_DB, //可取到的主数据库名称
Lists.newArrayList(DataSources.SLAVE_DB), //可存放多个从数据库
new RoundRobinMasterSlaveLoadBalanceAlgorithm() //均衡主从数据库
);
return MasterSlaveDataSourceFactory.createDataSource(map,masterSlaveRuleConfiguration, Maps.newHashMap(),new Properties());
}
// 配置事务
@Primary
@Bean
public DataSourceTransactionManager dataSourceTransactionManager(@Qualifier(DataSources.MASTER_DB) DataSource masterDb){
return new DataSourceTransactionManager(masterDb);
}
// 数据库工厂
@Primary
@Bean
public SqlSessionFactory sqlSessionFactory(@Qualifier(DataSources.MASTER_DB) DataSource masterDb) throws Exception{
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(masterDb);
bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath:mappers/*.xml"));
return bean.getObject();
}
@Primary
@Bean(name = "sqlSessionTemplate")
public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory){
return new SqlSessionTemplate(sqlSessionFactory);
}
} 部分对象说明：
1、DataSources：自己定义的数据库名称用于区分主数据库和从数据库（自己随便创建静态常量）
2、@ MapeprScan：注解第一个是mapper文件的地址用于写sql的xml文件路径（确保该地址正确的）
第二步：
主从配置写完接下来重头戏来了分库分表
import io.shardingsphere.api.config.rule.ShardingRuleConfiguration;
import io.shardingsphere.api.config.rule.TableRuleConfiguration;
import io.shardingsphere.api.config.strategy.StandardShardingStrategyConfiguration;
import io.shardingsphere.shardingjdbc.api.ShardingDataSourceFactory;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import javax.sql.DataSource;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.ConcurrentHashMap;
/**
* @ClassName: SeatDataSourceConfig
* @Description:
* @Author: TXW
* @Date: 2022/7/15
*/
@Configuration
@MapperScan(basePackages = "com.next.seatMapper",sqlSessionTemplateRef = "trainSeatSqlSessionTemplate")
public class SeatDataSourceConfig {
@Bean(name = DataSources.TRAIN_SEAT_DB_1)
@ConfigurationProperties(prefix = "spring.datasource-seat-1")
public DataSource trainSeatDB1(){
return DataSourceBuilder.create().build();
}
@Bean(name = DataSources.TRAIN_SEAT_DB_2)
@ConfigurationProperties(prefix = "spring.datasource-seat-2")
public DataSource trainSeatDB2(){
return DataSourceBuilder.create().build();
}
@Bean(name = DataSources.TRAIN_SEAT_DB_3)
@ConfigurationProperties(prefix = "spring.datasource-seat-3")
public DataSource trainSeatDB3(){
return DataSourceBuilder.create().build();
}
@Bean(name = DataSources.TRAIN_SEAT_DB_4)
@ConfigurationProperties(prefix = "spring.datasource-seat-4")
public DataSource trainSeatDB4(){
return DataSourceBuilder.create().build();
}
@Bean(name = DataSources.TRAIN_SEAT_DB_5)
@ConfigurationProperties(prefix = "spring.datasource-seat-5")
public DataSource trainSeatDB5(){
return DataSourceBuilder.create().build();
}
@Bean(name = "trainSeatShardingDataSource")
public DataSource trainSeatShardingDataSource(@Qualifier(DataSources.TRAIN_SEAT_DB_1) DataSource trainSeatDB1,
@Qualifier(DataSources.TRAIN_SEAT_DB_2) DataSource trainSeatDB2,
@Qualifier(DataSources.TRAIN_SEAT_DB_3) DataSource trainSeatDB3,
@Qualifier(DataSources.TRAIN_SEAT_DB_4) DataSource trainSeatDB4,
@Qualifier(DataSources.TRAIN_SEAT_DB_5) DataSource trainSeatDB5) throws SQLException {
ShardingRuleConfiguration shardingRuleConfiguration = new ShardingRuleConfiguration();
// 设置分库映射
Map map = new HashMap<>();
map.put(DataSources.TRAIN_SEAT_DB_1,trainSeatDB1);
map.put(DataSources.TRAIN_SEAT_DB_2,trainSeatDB2);
map.put(DataSources.TRAIN_SEAT_DB_3,trainSeatDB3);
map.put(DataSources.TRAIN_SEAT_DB_4,trainSeatDB4);
map.put(DataSources.TRAIN_SEAT_DB_5,trainSeatDB5);
// z设置表策略
TableRuleConfiguration configuration = new TableRuleConfiguration();
configuration.setLogicTable("train_seat");
configuration.setActualDataNodes(
DataSources.TRAIN_SEAT_DB_1+".train_seat_1,"+
DataSources.TRAIN_SEAT_DB_2+".train_seat_2,"+
DataSources.TRAIN_SEAT_DB_3+".train_seat_3,"+
DataSources.TRAIN_SEAT_DB_4+".train_seat_4,"+
DataSources.TRAIN_SEAT_DB_5+".train_seat_5,"+
DataSources.TRAIN_SEAT_DB_1+".train_seat_6,"+
DataSources.TRAIN_SEAT_DB_2+".train_seat_7,"+
DataSources.TRAIN_SEAT_DB_3+".train_seat_8,"+
DataSources.TRAIN_SEAT_DB_4+".train_seat_9,"+
DataSources.TRAIN_SEAT_DB_5+".train_seat_10"
);
// 设置分库策略
configuration.setDatabaseShardingStrategyConfig(new StandardShardingStrategyConfiguration(
"train_number_id",new TrainSeatDatabaseShardingAlgorithm()
));
// 设置分表策略
configuration.setTableShardingStrategyConfig(new StandardShardingStrategyConfiguration(
"train_number_id",new TrainSeatTableShardingAlgorithm()
));
shardingRuleConfiguration.getTableRuleConfigs().add(configuration);
return ShardingDataSourceFactory.createDataSource(map,shardingRuleConfiguration,new ConcurrentHashMap<>(),new Properties());
}
@Bean(name = "SeatDataSourceTransactionManager")
public DataSourceTransactionManager SeatDataSourceTransactionManager(@Qualifier("trainSeatShardingDataSource") DataSource trainSeatShardingDataSource){
return new DataSourceTransactionManager(trainSeatShardingDataSource);
}
@Bean(name = "trainSeatSqlSessionFactory")
public SqlSessionFactory seatTrainSeatSqlSessionFactory(@Qualifier("trainSeatShardingDataSource") DataSource trainSeatShardingDataSource) throws Exception {
SqlSessionFactoryBean bean =new SqlSessionFactoryBean();
bean.setDataSource(trainSeatShardingDataSource);
bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath:seatMappers/*.xml"));
return bean.getObject();
}
@Bean(name = "trainSeatSqlSessionTemplate")
public SqlSessionTemplate trainSeatSqlSessionTemplate (@Qualifier("trainSeatSqlSessionFactory") SqlSessionFactory trainSeatSqlSessionFactory){
return new SqlSessionTemplate(trainSeatSqlSessionFactory);
}
}
这个配置文件涉及到两个算法需要自己实现：
分库算法：
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import java.util.Collection;
/**
* @ClassName: TrainSeatDatabaseShardingAlgorithm
* @Description: 数据库分库算法实现
* @Author: TXW
* @Date: 2022/7/17
*/
public class TrainSeatDatabaseShardingAlgorithm implements PreciseShardingAlgorithm {
private final static String PREFIX="trainSeatDB";
private String determineDB(int val){
int db = val % 5;
if (db == 0){
db = 5;
}
return PREFIX + db;
}
@Override
public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) {
String actuallyDBName = determineDB(preciseShardingValue.getValue());
if (collection.contains(actuallyDBName)){
return actuallyDBName;
}
throw new IllegalArgumentException();
}
} 分表算法：
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import java.util.Collection;
/**
* @ClassName: TrainSeatTableShardingAlgorithm
* @Description: 自定义分库算法实现
* @Author: TXW
* @Date: 2022/7/17
*/
public class TrainSeatTableShardingAlgorithm implements PreciseShardingAlgorithm {
private final static String PREFIX="train_seat_";
private String determineTable(int val){
int db = val % 10;
if (db == 0){
db = 10;
}
return PREFIX + db;
}
@Override
public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) {
String actuallyTableName = determineTable(preciseShardingValue.getValue());
if (collection.contains(actuallyTableName)){
return actuallyTableName;
}
throw new IllegalArgumentException();
}
}
ok到此结束！！！！！！
总结：
分库分表配置还是比较有点麻烦的，暂时不可考虑多表查询带来的影响（es可以解决），就单单是根据每个公司的业务逻辑不同所分库分表的方式也有所不同，因此最主要的还是要看，业务上是按照什么进行分表的并且找到其中的关键字段进行区分。
分库分表涉及到一点算法，但是作为程序员对于这点算法应该还是基本上没有什么问题的。所以不管怎么说，还是要自己去梳理代码，根据自己的业务逻辑进行适当的修改，这样才能够实现目标需求，此处只是提供分库分表的思路和实现并不能代表拿去就能用。