mongodb 多数据源动态切换

有这样一种场景，公司的各个链路服务要进行压测，为了尽量得出准确的压测结果，直接在生产的服务上进行压测，但是有几个条件

压测的请求必须要标记识别，数据库，中间件这些数据的流转存储也必须作区分
mysql 有影子库，判断是压测的请求就直接操作影子库，除了 mysql, 可能还有 mongodb, es, hbase, redis这些数据都要落到影子库，压测完影子库删掉，不对生产库造成影响
重点是压测请求的识别与数据源的动态切换
请求的识别还包括在一个链路调用中，请求标记的识别跟透传
为了减少原有服务压测时候的改动，可以把这些影子库配置，动态切换的逻辑做成一个SDK，压测项目引入 SDK依赖就可以了

根据上面场景，现在来说明mongodb的多数据源的影子库，生产库动态切换怎么完成，mysql 已经有很多成熟的方案了，redis，mq这些有机会再研究.

1 压测请求的标记识别

这个应该说是相当简单了，在压测的请求加个header 标记, 过滤去里面判断到 header 标记是否存在，然后把标记放到 ThreadLocal 里面，后面的动态切换需要依赖这个副本变量

public class RequestTagFilter implements Filter {
    
    @Override
    public void init(FilterConfig filterConfig) throws ServletException {
        
    }
    
    @Override
    public void doFilter(ServletRequest request,
                         ServletResponse response,
                         FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;
        String header = httpServletRequest.getHeader("test");
        HeaderThreadLocal.setIsTestRequest(!StringUtils.isEmpty(header));
        chain.doFilter(request, response);
    }
    
    @Override
    public void destroy() {
        
    }
}

2 mongodb 多数据源配置

模拟生产的mongo配置.png

生产和影子库的配置文件是分开的

mongo影子库的配置.png

3 多数据源的切换逻辑的bean注入，只要加入这2个bean 就可以了

public class DynamicMongoDbFactory extends SimpleMongoDbFactory {

    //生产库的客户端与数据库
    private MongoClient mongoClientPro;
    private String databaseNamePro;

    //影子库的客户端与数据库
    private MongoClient mongoClientShodow;
    private String databaseNameShodow;
    
    private final boolean mongoInstanceCreated = false;
    private final PersistenceExceptionTranslator exceptionTranslator = null;

    private @Nullable WriteConcern writeConcern;

    public DynamicMongoDbFactory(MongoClient mongoClientPro, MongoClient mongoClientShodow,
                                 String databaseNamePro, String databaseNameShodow) {
        super(mongoClientPro, databaseNamePro);
        this.mongoClientPro = mongoClientPro;
        this.mongoClientShodow = mongoClientShodow;
        this.databaseNamePro = databaseNamePro;
        this.databaseNameShodow = databaseNameShodow;
    }
    @Override
    public void destroy() throws Exception {
        super.destroy();
    }
    @Override
    public MongoDatabase getDb() throws DataAccessException {
        //实现数据源动态切换的逻辑，判断是否是压测请求
        if (HeaderThreadLocal.isTestRequest()){
            return mongoClientShodow.getDatabase(databaseNameShodow);
        }
        return mongoClientPro.getDatabase(databaseNamePro);
    }
    @Override
    public MongoDatabase getDb(String dbName) throws DataAccessException {
        MongoDatabase mongoDatabase = mongoClientPro.getDatabase(dbName);
        if (mongoDatabase != null){
            return mongoDatabase;
        }
        return mongoClientPro.getDatabase(dbName);
    }
    @Override
    public PersistenceExceptionTranslator getExceptionTranslator() {
        return new MongoExceptionTranslator();
    }
    @SuppressWarnings("deprecation")
    @Override
    public DB getLegacyDb() {
        return mongoClientPro.getDB(databaseNamePro);
    }
}

@Configuration
@EnableAutoConfiguration
public class DynamicMongoRegister implements EnvironmentAware{

    private Environment environment;
    @Override
    public void setEnvironment(Environment environment) {
        this.environment = environment;
    }
    @Bean
    public DynamicMongoDbFactory simpleMongoDbFactory(){
        String mongoUrlPro = environment.getProperty("spring.data.mongodb.uri");
        String mongoUrlShadow = environment.getProperty("spring.data.mongodb.uri.shadow");
        //这里获取2个数据源的配置，生成2个数据源的客户端实例
        MongoClient mongoClientPro = new MongoClient(new MongoClientURI(mongoUrlPro, MongoClientOptions.builder()));
        MongoClient mongoClientShadow = new MongoClient(new MongoClientURI(mongoUrlShadow, MongoClientOptions.builder()));

        return new DynamicMongoDbFactory(mongoClientPro, mongoClientShadow, "produce", "shadow");
    }
    @Bean
    public MongoTemplate mongoTemplate(){
        return new MongoTemplate(simpleMongoDbFactory());
    }
}

4mongo数据库新建2个库，对应好库名，建好表

image.png

5 进行测试

使用springboot-data-mongodb的操作方式.png

测试用例调用.png