Debezium Configuration 设计文档
1. 核心设计理念
1.1 不可变配置
- 配置对象一旦创建就不能修改
- 所有修改操作都会返回新的配置对象
- 通过不可变性保证线程安全
- 使用
@Immutable
注解标记
1.2 组件化设计
Configuration
接口:定义配置的核心API
Field
类:描述配置字段的元数据和验证规则
CommonConnectorConfig
:所有连接器共享的基础配置
EmbeddedConfig
:嵌入式引擎特定的配置
2. 关键实现机制
2.1 构建器模式
Configuration config = Configuration.create()
.with("database.hostname", "localhost")
.with("database.port", 3306)
.build();
2.2 工厂方法
Configuration.from(properties)
Configuration.empty()
Configuration.copy(existingConfig)
2.3 配置组合
Configuration subset = config.subset("database.", true);
Configuration derived = Configuration.create()
.withDefault(baseConfig)
.with("new.property", "value")
.build();
3. 配置管理特性
3.1 多实例支持
- 允许创建多个配置实例
- 不同组件使用独立配置
- 配置之间相互隔离
3.2 类型安全
3.3 验证机制
Field HOST = Field.create("database.host")
.description("Database host")
.required()
.build();
List<ConfigValue> problems = config.validate(
new HashSet<>(Arrays.asList(HOST, PORT, USERNAME))
);
4. 使用模式
4.1 依赖注入
public class DatabaseConnector {
private final Configuration config;
public DatabaseConnector(Configuration config) {
this.config = config;
validate(config);
}
}
4.2 配置转换
Configuration mapped = config.mapped((key, value) ->
transformValue(key, value));
Configuration resolved = config.withReplacedVariables(
variableName -> lookupValue(variableName)
);
5. 设计优势
5.1 线程安全
- 不可变对象天生线程安全
- 无需同步机制
- 配置更新原子性
5.2 可测试性
- 配置对象易于模拟
- 支持测试配置替换
- 验证机制可测试
5.3 灵活性
5.4 可维护性
6. 最佳实践
6.1 配置创建
- 使用构建器模式创建配置
- 设置必要的验证规则
- 提供合理的默认值
6.2 配置使用
- 通过依赖注入传递配置
- 及早进行配置验证
- 使用类型安全的访问方法
6.3 配置管理
- 按组件划分配置
- 利用配置继承复用通用设置
- 保持配置的不可变性
7. 源码分析
7.1 Configuration 接口
@Immutable
public interface Configuration {
String getString(String key);
int getInteger(String key, int defaultValue);
boolean getBoolean(String key, boolean defaultValue);
static Configuration from(Properties props);
static Configuration empty();
Configuration subset(String prefix, boolean stripPrefix);
Configuration withDefault(Configuration defaults);
}
7.2 Field 类
public class Field {
private final String name;
private final String description;
private final boolean required;
private final Object defaultValue;
public static Builder create(String name) {
return new Builder(name);
}
public ValidationOutput validate(Configuration config) {
}
}
7.3 CommonConnectorConfig 类
public class CommonConnectorConfig {
private final Configuration config;
public CommonConnectorConfig(Configuration config) {
this.config = config;
validate();
}
public static final Field TOPIC_PREFIX = Field.create("topic.prefix")
.required()
.description("Topic prefix for all generated events");
public static final Field MAX_BATCH_SIZE = Field.create("max.batch.size")
.defaultValue(2048)
.description("Maximum size of each batch of source records.");
}
8. 配置示例
8.1 MySQL 连接器配置
Configuration mysqlConfig = Configuration.create()
.with("connector.class", "io.debezium.connector.mysql.MySqlConnector")
.with("database.hostname", "mysql1")
.with("database.port", "3306")
.with("database.user", "debezium")
.with("database.password", "dbz")
.with("database.server.id", "1")
.with("topic.prefix", "mysql")
.with("schema.history.internal.kafka.topic", "schema-changes.mysql")
.build();
8.2 PostgreSQL 连接器配置
Configuration postgresConfig = Configuration.create()
.with("connector.class", "io.debezium.connector.postgresql.PostgresConnector")
.with("database.hostname", "postgres1")
.with("database.port", "5432")
.with("database.user", "debezium")
.with("database.password", "dbz")
.with("database.dbname", "inventory")
.with("topic.prefix", "postgres")
.build();
8.3 配置组合示例
Configuration baseConfig = Configuration.create()
.with("topic.prefix", "myapp")
.with("database.history.kafka.bootstrap.servers", "kafka:9092")
.build();
Configuration mysqlConfig = Configuration.create()
.withDefault(baseConfig)
.with("connector.class", "io.debezium.connector.mysql.MySqlConnector")
.with("database.hostname", "mysql1")
.build();
Configuration postgresConfig = Configuration.create()
.withDefault(baseConfig)
.with("connector.class", "io.debezium.connector.postgresql.PostgresConnector")
.with("database.hostname", "postgres1")
.build();