我们做的是一个分布式数据库运维平台,项目会配置自己的数据源,同时因为是数据库运维平台,可以动态接入新的数据库集群,需要到这些集群节点系统表查询一些运维数据。接手项目前,他们的做法是手动创建JDBC连接,拼接sql语句去查询。
实现如下:
MgrHost host = new MgrHost();
StringBuilder querySql = new StringBuilder();
querySql.append("select t.hostaddr, t.hostagentport");
querySql.append(" from mgr_host t");
querySql.append(" where t.oid = ").append(node.getNodehost());
try (Connection conn = clusterConnService.createSpecifiedClusterConn(clusterInfo);
Statement statement = conn.createStatement();
ResultSet resultSet = statement.executeQuery(querySql.toString())) {
//获取连接
while (resultSet.next()) {
host.setHostaddr(resultSet.getString(1));
host.setHostagentport(resultSet.getInt(2));
}
} catch (Exception e) {
// 查询节点失败
logger.error("query mgr_host failed", e);
throw new com.ai.dbops.api.commons.DBOpsException("query mgr_host failed");
}
public Connection createSpecifiedClusterConn(ClusterInfo clusterInfo) throws ClassNotFoundException, SQLException {
logger.debug("start create specified connection");
String driverClassName = "org.postgresql.Driver";
String url = "jdbc:postgresql://" + clusterInfo.getMgrAddr() + "/"
+ antDbConfig.getNodeConnectDatabase() + "?useUnicode=true";
Class.forName(driverClassName);
Connection conn = DriverManager.getConnection(url, clusterInfo.getClusterConnInfo().getDbUser(), clusterInfo.getClusterConnInfo().getDbPasswd());
try (Statement stmt = conn.createStatement()) {
stmt.execute("set command_mode to sql");
} catch (Exception e) {
try {
if (conn != null) {
conn.close();
}
} catch (Exception e1) {
logger.error(ExceptionUtils.getStackTrace(e1));
}
}
return conn;
}
- 手动创建jdbc连接,繁琐复杂,忘记关闭连接还可能造成数据库连接滥用
- Sql语句都是在代码里硬编码,影响代码整洁可读性
- 可能会无限创建连接,造成数据库连接过多;每次都需要新建连接,连接耗时影响性能。
针对以上问题,解决方案是利用数据源来管理连接,可解决1 3两个问题;通过实现spring提供的AbstractRoutingDataSource接口配置动态数据源,并植入MyBatis,在mapper写sql ,可以解决问题2。
@Component
public class DynamicDataSource extends AbstractRoutingDataSource {
@Resource(name = "dataSource")
private DataSource sqlModeDataSource;
//1.首先自定义DynamicDataSourceContextHolder来持有dataSource的key,
//这边是ThreadLocal实现,多线程线程隔离的,下面贴实现。
//2.查看AbstractRoutingDataSource源码, getConnection()方法会通过这个方法决定获取哪个数据源,这个方法是动态数据源的关键
@Override
public Object determineCurrentLookupKey() {
return DynamicDataSourceContextHolder.getDataSourceKey();
}
//这里是系统默认数据源 初始化时注入进来
@PostConstruct
public void initDefaultDataSource() {
log.info("dynamicDataSource postConstruct ...");
Map<Object, Object> targetDataSource = new HashMap<>();
targetDataSource.put(DynamicDataSourceContextHolder.DEFAULT_DADA_SOURCE, sqlModeDataSource);
this.setTargetDataSources(targetDataSource);
}
//系统动态添加数据源进来,通过这个方法,并缓存对应的key
public void addDataSource(String key, DataSource dataSource) {
Map<Object, Object> targetDataSources = new HashMap<>(this.getResolvedDataSources());
targetDataSources.put(key, dataSource);
this.setTargetDataSources(targetDataSources);
DynamicDataSourceContextHolder.addDataSourceKey(key);
this.afterPropertiesSet();
}
}
DynamicDataSourceContextHolder实现,很简单,通过ThreadLocal实现
public class DynamicDataSourceContextHolder {
private static final ThreadLocal<String> contextHolder = ThreadLocal.withInitial(() -> DEFAULT_DATA_SOURCE);
/**
* 数据源的 key集合,用于切换时判断数据源是否存在
*/
private static final CopyOnWriteArrayList<Object> dataSourceKeys = new CopyOnWriteArrayList<>();
/**
* 切换数据源
*
* @param key key
*/
public static void setDataSourceKey(String key) {
contextHolder.set(key);
}
/**
* 获取数据源
*
* @return key
*/
public static String getDataSourceKey() {
return contextHolder.get();
}
/**
* 重置数据源
*/
public static void clearDataSourceKey() {
contextHolder.remove();
}
/**
* 判断是否包含数据源
*
* @param key 数据源key
* @return true
*/
public static boolean containsDataSourceKey(String key) {
return dataSourceKeys.contains(key);
}
/**
* 添加数据源keys
*
* @param keys keys
* @return true
*/
public static boolean addDataSourceKeys(Collection<String> keys) {
return dataSourceKeys.addAll(keys);
}
public static void addDataSourceKey(String key) {
dataSourceKeys.add(key);
}
}
这边就是MyBatis的配置,把sqlSessionFactory的数据源配置成动态数据源
@Bean("sqlSessionFactory")
public SqlSessionFactory antdbSqlSessionFactory(@Qualifier("dynamicDataSource") DataSource dataSource)
throws Exception {
}
ClusterInfo clusterInfo = new ClusterInfo();
clusterInfo.setClusterId(111);
XXXNode node = DataSourceUtils.execute(clusterInfo, () -> {
xxxMapper.queryNodes(param1, parma2);
})
下面看一下DataSourceUtils的实现,分四步:
@Component
@Slf4j
public class DataSourceUtils {
private static DataSourceProperties properties;
private static DynamicDataSource dynamicDataSource;
private static ClusterConnService clusterConnService;
@PostConstruct
private void init() {
properties = ApplicationContextUtils.getBean(DataSourceProperties.class);
dynamicDataSource = ApplicationContextUtils.getBean("dynamicDataSource");
clusterConnService = ApplicationContextUtils.getBean(ClusterConnService.class);
}
public static <T> T execute(ClusterInfo clusterInfo, Supplier<T> supplier) {
//如果没提供集群id 则按默认数据源查询
if (Objects.isNull(clusterInfo) || Objects.isNull(clusterInfo.getClusterId())) {
return supplier.get();
}
//这是去获取数据源,如果没有被缓存,就初始化数据源
String datasourceKey = DynamicDataSourceContextHolder.DS_KEY_PREFIX + clusterInfo.getClusterId();
if (!DynamicDataSourceContextHolder.containsDataSourceKey(datasourceKey)) {
synchronized (DataSourceUtils.class) {
if (!DynamicDataSourceContextHolder.containsDataSourceKey(datasourceKey)) {
initDataSource(datasourceKey, clusterInfo, true);
}
}
}
return doExec(supplier, datasourceKey);
}
//初始化完数据源,就通过DynamicDataSourceContextHolder设置数据源的key,然后执行sql语句
private static <T> T doExec(Supplier<T> supplier, String datasourceKey) {
log.info("switch to dataSource[{}]", datasourceKey);
DynamicDataSourceContextHolder.setDataSourceKey(datasourceKey);
T t = supplier.get();
DynamicDataSourceContextHolder.clearDataSourceKey();
log.info("dataSource[{}] finished, switch to default", datasourceKey);
return t;
}
//这里就是初始化数据源,并且根据对应的key缓存到动态数据源里
private static void initDataSource(String datasourceKey, ClusterInfo clusterInfo, boolean sqlMode) {
String address = clusterInfo.getMgrAddr();
String username = clusterInfo.getClusterConnInfo().getDbUser();
String password = clusterInfo.getClusterConnInfo().getDbPasswd();
String dbName = antDbConfig.getNodeConnectDatabase();
log.info("init dataSource , poolName:{}", datasourceKey);
String url = "jdbc:postgresql://" + address + "/" + dbName + "?useUnicode=true";
Class<? extends HikariDataSource> clazz = sqlMode ? SqlModeDataSource.class : HikariDataSource.class;
HikariDataSource dataSource = properties.initializeDataSourceBuilder()
.type(clazz)
.driverClassName("org.postgresql.Driver")
.url(url)
.username(username)
.password(password)
.build();
dataSource.setPoolName(datasourceKey);
dynamicDataSource.addDataSource(datasourceKey, dataSource);
}
}
上面切换完数据源后,有人可能还会有疑问为什么这样执行mapper就会使用我们切过的数据源执行?
看过Mybatis源码的应该可以理解,这里mapper是Mybatis动态代理类,调用mapper方法最终是通过刚才配置的数据源getConnection来获取数据库连接,动态数据源getConnection方法上面我们解释过了,是通过我们动态改变key来获取的。