解决sharding-sphere强制扫描表结构的方法
解决sharding-sphere强制扫描表结构的方法
##解决sharding-sphere强制扫描表结构的方法
- 在sharding-sphere3.0.0的版本发布之后会强制校验分库分表的表结构的结构是否一致，
如果不一致会报错：
Caused by: io.shardingsphere.core.exception.ShardingException: Cannot get uniformed table structure for `xxxxx`. The different meta data of actual tables are as follows:
TableMetaData(columnMetaData=[ColumnMetaData(columnName=order_id_a,......... ])
TableMetaData(columnMetaData=[ColumnMetaData(columnName=order_id, .........]).
at io.shardingsphere.core.metadata.table.executor.TableMetaDataLoader.checkUniformed(TableMetaDataLoader.java:148)
at io.shardingsphere.core.metadata.table.executor.TableMetaDataLoader.load(TableMetaDataLoader.java:71)
at io.shardingsphere.core.metadata.table.executor.TableMetaDataInitializer.loadShardingTables(TableMetaDataInitializer.java:73)
at io.shardingsphere.core.metadata.table.executor.TableMetaDataInitializer.load(TableMetaDataInitializer.java:62)
at io.shardingsphere.core.metadata.ShardingMetaData.(ShardingMetaData.java:46)
at io.shardingsphere.shardingjdbc.jdbc.core.ShardingContext.(ShardingContext.java:63)
at io.shardingsphere.shardingjdbc.jdbc.core.datasource.ShardingDataSource.getShardingContext(ShardingDataSource.java:85)
at io.shardingsphere.shardingjdbc.jdbc.core.datasource.ShardingDataSource.(ShardingDataSource.java:65)
at io.shardingsphere.shardingjdbc.api.ShardingDataSourceFactory.createDataSource(ShardingDataSourceFactory.java:51)
-
在3.1.0的发布版本中增加了check.table.metadata.enabled配置，默认不检查表结构是否一致。
https://shardingsphere.apache.org/document/legacy/3.x/document/en/manual/sharding-jdbc/configuration/config-java/ -
升级sharding-jdbc
A：升级jar pom.xml
io.shardingsphere
sharding-jdbc-core
3.0.0
修改为：
io.shardingsphere
sharding-jdbc-core
3.1.0
B: 更改包结构
import io.shardingsphere.api.config.ShardingRuleConfiguration;
import io.shardingsphere.api.config.TableRuleConfiguration;
修改为：
import io.shardingsphere.api.config.rule.ShardingRuleConfiguration;
import io.shardingsphere.api.config.rule.TableRuleConfiguration;
- 版本局限
sharding-sphere3.1.0虽然解决了强制扫描表结构的问题，但是引出了其他的问题，经过测试如果在分库分表的数据库中，查询单表的数据会导致shardjing-sphere的路由失败，设置DefaultDataSourceName也不会起作用，通过阅读源码推测是sharding-sphere的bug，所以如果有查询单表的sql的应用先不要升级；
ps：官方发布的4.0.x解决了改问题，但是在maven的中央仓库还没有搜索到，可以耐心等待。 - 暴力方案
通过阅读码发现是TableMetaDataLoader类在处理扫描表格结构的功能，可以在工程中将该类拷贝，注释扫描功能，对原有功能没有影响。
public final class TableMetaDataLoader {
private final ShardingDataSourceMetaData shardingDataSourceMetaData;
private final ShardingExecuteEngine executeEngine;
private final TableMetaDataConnectionManager connectionManager;
private final int maxConnectionsSizePerQuery;
/**
* Load table meta data.
*
* @param logicTableName logic table name
* @param shardingRule sharding rule
* @return table meta data
* @throws SQLException SQL exception
*/
public TableMetaData load(final String logicTableName, final ShardingRule shardingRule) throws SQLException {
List actualTableMetaDataList = load(shardingRule.getTableRuleByLogicTableName(logicTableName).getDataNodeGroups(), shardingRule.getShardingDataSourceNames());
// todo 注释强制扫描表结构的代码
// checkUniformed(logicTableName, actualTableMetaDataList);
return actualTableMetaDataList.iterator().next();
}
private List load(final Map> dataNodeGroups, final ShardingDataSourceNames shardingDataSourceNames) throws SQLException {
return executeEngine.groupExecute(getDataNodeGroups(dataNodeGroups), new ShardingGroupExecuteCallback() {
@Override
public Collection execute(final Collection dataNodes, final boolean isTrunkThread) throws SQLException {
String dataSourceName = dataNodes.iterator().next().getDataSourceName();
DataSourceMetaData dataSourceMetaData = shardingDataSourceMetaData.getActualDataSourceMetaData(dataSourceName);
String catalog = null == dataSourceMetaData ? null : dataSourceMetaData.getSchemeName();
return load(shardingDataSourceNames.getRawMasterDataSourceName(dataSourceName), catalog, dataNodes);
}
});
}
private Collection load(final String dataSourceName, final String catalog, final Collection dataNodes) throws SQLException {
Collection result = new LinkedList<>();
try (Connection connection = connectionManager.getConnection(dataSourceName)) {
for (DataNode each : dataNodes) {
result.add(new TableMetaData(
isTableExist(connection, catalog, each.getTableName()) ? getColumnMetaDataList(connection, catalog, each.getTableName()) : Collections.emptyList()));
}
}
return result;
}
private Collection> getDataNodeGroups(final Map> dataNodeGroups) {
Collection> result = new LinkedList<>();
for (Entry> entry : dataNodeGroups.entrySet()) {
result.addAll(getDataNodeGroups(entry.getValue()));
}
return result;
}
private Collection> getDataNodeGroups(final List dataNodes) {
Collection> result = new LinkedList<>();
for (List each : Lists.partition(dataNodes, Math.max(dataNodes.size() / maxConnectionsSizePerQuery, 1))) {
result.add(new ShardingExecuteGroup<>(each));
}
return result;
}
private boolean isTableExist(final Connection connection, final String catalog, final String actualTableName) throws SQLException {
try (ResultSet resultSet = connection.getMetaData().getTables(catalog, null, actualTableName, null)) {
return resultSet.next();
}
}
private List getColumnMetaDataList(final Connection connection, final String catalog, final String actualTableName) throws SQLException {
List result = new LinkedList<>();
Collection primaryKeys = getPrimaryKeys(connection, catalog, actualTableName);
try (ResultSet resultSet = connection.getMetaData().getColumns(catalog, null, actualTableName, null)) {
while (resultSet.next()) {
String columnName = resultSet.getString("COLUMN_NAME");
String columnType = resultSet.getString("TYPE_NAME");
result.add(new ColumnMetaData(columnName, columnType, primaryKeys.contains(columnName)));
}
}
return result;
}
private Collection getPrimaryKeys(final Connection connection, final String catalog, final String actualTableName) throws SQLException {
Collection result = new HashSet<>();
try (ResultSet resultSet = connection.getMetaData().getPrimaryKeys(catalog, null, actualTableName)) {
while (resultSet.next()) {
result.add(resultSet.getString("COLUMN_NAME"));
}
}
return result;
}
private void checkUniformed(final String logicTableName, final List actualTableMetaDataList) {
final TableMetaData sample = actualTableMetaDataList.iterator().next();
for (TableMetaData each : actualTableMetaDataList) {
if (!sample.equals(each)) {
throw new ShardingException("Cannot get uniformed table structure for `%s`. The different meta data of actual tables are as follows:\n%s\n%s.", logicTableName, sample, each);
}
}
}
}