三 ShardingSphere分库分表实战

1 概念

        逻辑表：逻辑意义上统一的表

        真实表：数据库中真正存在的表

        数据节点：数据分片的最小单元。由数据库、数据源组成

        分片键：用于分片的数据库字段。需要有足够的分辨度

        分片算法：用于分片的算法，支持=、between、in

        分片策略：分片键+分片算法。在ShardingJDBC中一般采用基于Groovy表达式的inline分片策略，如user_$->{user_id%8}，通过模8算法拆成8张表，user_0 -- user_7

2 实战

2.1 数据库

database：lx、lx2

table：course，course_1，course_2

DROP TABLE IF EXISTS `course`;
CREATE TABLE `course`  (
  `cid` bigint NOT NULL,
  `cname` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NULL DEFAULT NULL,
  `user_id` bigint NULL DEFAULT NULL,
  `cstatus` tinyint(1) NULL DEFAULT NULL,
  PRIMARY KEY (`cid`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_0900_ai_ci ROW_FORMAT = Dynamic;

SET FOREIGN_KEY_CHECKS = 1;

2.2 spring boot

2.2.1 代码

2.2.1.1 依赖

        
            org.springframework.boot
            spring-boot-starter
            2.4.5
        
        
            org.springframework.boot
            spring-boot-starter-test
            2.4.1
        
        
            mysql
            mysql-connector-java
            8.0.19
        
        
            org.springframework.boot
            spring-boot-starter-jdbc
            2.4.5
        
        
            com.alibaba
            druid
            1.1.23
        
        
            com.baomidou
            mybatis-plus-boot-starter
            3.3.2
        
        
            org.apache.shardingsphere
            sharding-jdbc-spring-boot-starter
            4.1.1
        
        
            org.projectlombok
            lombok
            1.18.26
        
        
            junit
            junit
            RELEASE
            test
        
    

2.2.1.2 项目结构

2.2.1.3 配置

单库配置：

# 虚拟数据库
spring.shardingsphere.datasource.names=m1
# 配置jdbc属性
spring.shardingsphere.datasource.m1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.m1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.m1.url=jdbc:mysql://192.168.31.31:3306/lx
spring.shardingsphere.datasource.m1.username=root
spring.shardingsphere.datasource.m1.password=12345
# 虚拟表tables.course和真实表course_$->{1..2}的对应关系
spring.shardingsphere.sharding.tables.course.actual-data-nodes=m1.course_$->{1..2}
# 主键生成策略
spring.shardingsphere.sharding.tables.course.key-generator.column=cid
spring.shardingsphere.sharding.tables.course.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.course.key-generator.props.work.id=1
# 分片算法
## 分片键
spring.shardingsphere.sharding.tables.course.table-strategy.inline.sharding-column=cid
## 分片算法
spring.shardingsphere.sharding.tables.course.table-strategy.inline.algorithm-expression=course_$->{cid%2+1}
#打印实时日志
spring.shardingsphere.props.sql.show=true

分析：

1 表course 是否可删除？

        可以删除，配置中course只是逻辑表，真实表是（course_1、course_2）

2 可以不设置虚拟表吗？

        tables.course虚拟表可以去掉，但最好加上，逻辑表和真实表的关系显而易见

3 shardingSphere在其中做的事情？

        只负责将LogicSql->ActualSql中，并不关心成功与否

2.2.1.4 实体

@Data
public class Course {
    @TableId
    private Long cid;
    @TableField(value = "user_id")
    private Long userId;
    private String cname;
    private Integer cstatus;
}

2.2.1.5 mapper

@Mapper
public interface CourseMapper extends BaseMapper {
}

2.2.1.6 test

@RunWith(SpringRunner.class)
@SpringBootTest
public class ShardingTest {

    @Resource
    private CourseMapper courseMapper;

    @Test
    public void addCourse() {
        for (int i = 0; i < 10; i++) {
            Course course = new Course();
            course.setCname("java" + i);
            course.setCstatus(1);
            course.setUserId(1000L);
            courseMapper.insert(course);
        }
    }
}

2.2.2 执行结果

         在执行插入操作时，LogicSql针对逻辑表course，ActualSql针对实际表（course_1、course_2）；插入的结果按照分片策略（分片键：cid + 分片算法： [course_$->{cid%2+1}]）

2.2.3 扩展

2.2.3.1 多库配置

# 虚拟数据库
spring.shardingsphere.datasource.names=m1,m2
# 配置jdbc属性
spring.shardingsphere.datasource.m1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.m1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.m1.url=jdbc:mysql://192.168.31.31:3306/lx
spring.shardingsphere.datasource.m1.username=root
spring.shardingsphere.datasource.m1.password=12345

spring.shardingsphere.datasource.m2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.m2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.m2.url=jdbc:mysql://192.168.31.31:3306/lx2
spring.shardingsphere.datasource.m2.username=root
spring.shardingsphere.datasource.m2.password=12345

# 虚拟表tables.course和真实表course_$->{1..2}的对应关系
spring.shardingsphere.sharding.tables.course.actual-data-nodes=m$->{1..2}.course_$->{1..2}
# 主键生成策略
spring.shardingsphere.sharding.tables.course.key-generator.column=cid
spring.shardingsphere.sharding.tables.course.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.course.key-generator.props.work.id=1
# 库分片算法
## 分片键
spring.shardingsphere.sharding.tables.course.database-strategy.inline.sharding-column=cid
## 分片算法
spring.shardingsphere.sharding.tables.course.database-strategy.inline.algorithm-expression=m$->{cid%2+1}
# 表分片算法
## 分片键
spring.shardingsphere.sharding.tables.course.table-strategy.inline.sharding-column=cid
## 分片算法
spring.shardingsphere.sharding.tables.course.table-strategy.inline.algorithm-expression=course_$->{(cid%4+1).intdiv(2)}
#打印实时日志
spring.shardingsphere.props.sql.show=true

2.2.3.2 范围查询

# 库分片算法
## 分片键
spring.shardingsphere.sharding.tables.course.database-strategy.standard.sharding-column=cid
## 分片算法
spring.shardingsphere.sharding.tables.course.database-strategy.standard.range-algorithm-class-name=com.roy.algorithm.MyRangeDsShardingAlgorithm
spring.shardingsphere.sharding.tables.course.database-strategy.standard.precise-algorithm-class-name=com.roy.algorithm.MyPreciseDsShardingAlgorithm
# 表分片算法
## 分片键
spring.shardingsphere.sharding.tables.course.table-strategy.standard.sharding-column=cid
## 分片算法
spring.shardingsphere.sharding.tables.course.table-strategy.standard.range-algorithm-class-name=com.roy.algorithm.MyRangeShardingAlgorithm
spring.shardingsphere.sharding.tables.course.table-strategy.standard.precise-algorithm-class-name=com.roy.algorithm.MyPreciseShardingAlgorithm

public class MyPreciseDsShardingAlgorithm implements PreciseShardingAlgorithm {
    @Override
    public String doSharding(Collection avaliableTargetNames/*真实表集合*/, PreciseShardingValue shardingValue/*精确查询的条件*/) {
        //实现按照 = 或 IN 进行精确分片
        // 例如 select * from course where cid = 1 or cid in (1,3,5)
        // 实现 course_$->{cid%2+1} 分表策略
        BigInteger shardingValueB = BigInteger.valueOf(shardingValue.getValue());
        BigInteger resB = shardingValueB.mod(new BigInteger("2")).add(new BigInteger("1"));
        String key = "m" + resB;
        if (avaliableTargetNames.contains(key)) {
            return key;
        }
        throw new UnsupportedOperationException("route:" + key + "is not support");
    }
}

public class MyRangeDsShardingAlgorithm implements RangeShardingAlgorithm /*主键类型*/ {
    @Override
    public Collection doSharding(Collection avaliableTargetNames/*真实表集合*/, RangeShardingValue shardingValue/*上限、下限*/) {
        Long lowerEndpoint = shardingValue.getValueRange().lowerEndpoint();
        Long upperEndpoint = shardingValue.getValueRange().upperEndpoint();
        return avaliableTargetNames;
    }
}

public class MyPreciseShardingAlgorithm implements PreciseShardingAlgorithm {
    @Override
    public String doSharding(Collection avaliableTargetNames/*真实表集合*/, PreciseShardingValue shardingValue/*精确查询的条件*/) {
        //实现按照 = 或 IN 进行精确分片
        // 例如 select * from course where cid = 1 or cid in (1,3,5)
        // 实现 course_$->{cid%2+1} 分表策略
        BigInteger shardingValueB = BigInteger.valueOf(shardingValue.getValue());
        BigInteger resB = shardingValueB.mod(new BigInteger("2")).add(new BigInteger("1"));
        String key = shardingValue.getLogicTableName() + "_" + resB;
        if (avaliableTargetNames.contains(key)) {
            return key;
        }
        throw new UnsupportedOperationException("route:" + key + "is not support");
    }
}

public class MyRangeShardingAlgorithm implements RangeShardingAlgorithm /*主键类型*/ {
    @Override
    public Collection doSharding(Collection avaliableTargetNames/*真实表集合*/, RangeShardingValue shardingValue/*上限、下限*/) {
        Long lowerEndpoint = shardingValue.getValueRange().lowerEndpoint();
        Long upperEndpoint = shardingValue.getValueRange().upperEndpoint();
        return Arrays.asList(shardingValue.getLogicTableName() + "_1", shardingValue.getLogicTableName() + "_2");
    }
}

        分库分表会对查询条件进行限制，一个应用中既可以有普通查询、范围查询等，而在分库分表中，要么是inline的普通查询、要么是standard + 算法类的范围查询等，所以分库分表能不用就不用，使用后就要考虑查询条件被限制