在分布式的系统里,数据一致性往往是首先关注且最难解决的部分。市面上也有很多分布式事务框架,比如seata
、hmily
等,但貌似业界并没有大规模的使用某一框架,不像Dubbo
、Spring Cloud
那样使用比较集中。这是因为分布式事务更需要从项目实际的业务情况考虑,这些框架实现的理论无非就是基于2PC、3PC、TCC、Saga等。这里介绍基于Saga
的一种实现思路。
Saga
:适用于长活事务场景,解决长活事务长时间阻塞数据库,以及像外部系统无法提供TCC所需要的接口。通过每个事务执行时本地提交,回滚时向前恢复或向后恢复,来达到不阻塞的目的。缺点是没有隔离性。
事务发起者
:主程序
事务接收者
:事务中的执行步骤
事务管理器
:对事务的定义、事务的运行状态和运行流程进行管理
发起事务的时候,需要知道事务中的步骤,这时候就需要提前对事务的定义进行管理,作用类似于注册中心,需要提前注册才能在调用的时候进行感知。
事务定义管理是事务管理器
功能,这里实现的思路是:事务发起者和事务接收者服务启动时,将事务的定义保存在本地对象中,事务管理器根据注册中心的实例变化事件,感知到事务发起者和接收者的上线,通过restful api
将他们本地对象数据保存到数据库。从而实现事务的管理。
事务发起者和接收者启动时,将事务定义保存本地对象
这里使用BeanPostProcessor
在初始化Bean之后检查是否有对应的事务注解,如果有的话将事务保存在本地对象
/**
* Bean初始化之后将事务保存到本地对象
* @author Tarzan写bug
* @since 2022/11/03
*/
public class SagaPropertyDataProcessor implements BeanPostProcessor {
private SagaPropertyData propertyData;
public SagaPropertyDataProcessor(SagaPropertyData propertyData) {
this.propertyData = propertyData;
}
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
Method[] methods = ReflectionUtils.getAllDeclaredMethods(bean.getClass());
if (methods != null) {
for (Method method : methods) {
addSaga(method);
addSagaTask(method);
}
}
return bean;
}
private void addSaga(Method method) {
Saga saga = AnnotationUtils.findAnnotation(method, Saga.class);
SagaProperty sagaProperty = new SagaProperty(saga.code(), saga.description());
addInputSchema(sagaProperty, saga);
}
private void addSagaTask(Method method) {
SagaTask sagaTask = AnnotationUtils.findAnnotation(method, SagaTask.class);
SagaTaskProperty sagaTaskProperty
= new SagaTaskProperty(sagaTask.code(), sagaTask.sagaCode(), sagaTask.seq(), sagaTask.description());
addOutputSchema(sagaTaskProperty, sagaTask);
}
private void addInputSchema(SagaProperty property, Saga saga) {
property.setInputSchema(saga.inputSchema());
property.setInputSchemaClass(saga.inputSchemaClass());
}
private void addOutputSchema(SagaTaskProperty property, SagaTask sagaTask) {
property.setOutputSchema(sagaTask.outputSchema());
property.setOutputSchemaClass(sagaTask.outputSchemaClass());
}
}
提供内部restful api给事务管理器获取本地对象
/**
* 获取本地事务对象接口
* @author Tarzan写bug
* @since 2022/11/04
*/
@RestController
public class SagaPropertyDataEndpoint {
private SagaPropertyData sagaPropertyData;
public SagaPropertyDataEndpoint(SagaPropertyData sagaPropertyData) {
this.sagaPropertyData = sagaPropertyData;
}
@GetMapping(value = "/sagas", produces = {MediaType.APPLICATION_JSON_VALUE})
public SagaPropertyData getSagaPropertyData() {
return sagaPropertyData;
}
}
事务管理器利用注册中心实例变化,监听到实例上线后,调用实例的接口获取事务定义,并将事务定义保存到数据库
/**
* 服务事件监听
* @author Tarzan写bug
* @since 2022/11/04
*/
public class ServiceEventObserver implements ApplicationListener<ServiceChangeEvent> {
@Override
public void onApplicationEvent(ServiceChangeEvent serviceChangeEvent) {
// 监听到实例上线则将事务加入到数据库中
}
}
向事务管理器发起预提交,然后执行本地逻辑,这两步在本地事务中保持一致性。当出现异常时,进行事务回滚并向事务管理器发起取消事务信号。
/**
* 创建事务DTO
* @author Tarzan写bug
* @since 2022/11/04
*/
public class SagaInstanceDTO implements Serializable {
private static final Long serialVersionUID = 1L;
private String service;
private String code;
private String inputSchema;
private String uuid;
// getset...
}
/**
* 跟事务管理器通讯
* @author Tarzan写bug
* @date 2022/11/04
*/
public interface SagaClient {
/**
* 预提交
* @param dto
*/
void preConfirmSaga(SagaInstanceDTO dto);
/**
* 提交
* @param dto
*/
void confirmSaga(SagaInstanceDTO dto);
/**
* 取消Saga
* @param uuid
*/
void cancelSaga(String uuid);
}
/**
* 启动saga传参
* @author Tarzan写bug
* @since 2022/11/04
*/
public class StartSagaBuilder {
private SagaInstanceDTO instanceDTO;
public static StartSagaBuilder newBuilder() {
return new StartSagaBuilder();
}
public StartSagaBuilder withService(String service) {
instanceDTO.setService(service);
return this;
}
public StartSagaBuilder withCode(String code) {
instanceDTO.setCode(code);
return this;
}
public StartSagaBuilder withInputSchema(String inputSchema) {
instanceDTO.setInputSchema(inputSchema);
return this;
}
public StartSagaBuilder withUuid(String uuid) {
instanceDTO.setUuid(uuid);
return this;
}
/**
* 预提交参数
* @return
*/
public SagaInstanceDTO preBuild() {
return instanceDTO;
}
/**
* 提交参数
* @return
*/
public SagaInstanceDTO confirmBuild() {
return instanceDTO;
}
}
/**
* 事务发起者操作接口
* @author Tarzan写bug
* @date 2022/11/04
*/
public interface SagaLaunch {
/**
* 发起事务
* @param builder
* @param consumer
*/
void apply(StartSagaBuilder builder, Consumer<StartSagaBuilder> consumer);
}
/**
* 事务发起者操作实现类
* @author Tarzan写bug
* @since 2022/11/04
*/
public class SagaLaunchImpl implements SagaLaunch {
private PlatformTransactionManager transactionManager;
private SagaClient sagaClient;
private String service;
public SagaLaunchImpl(PlatformTransactionManager transactionManager, SagaClient sagaClient, String service) {
this.transactionManager = transactionManager;
this.sagaClient = sagaClient;
this.service = service;
}
@Override
public void apply(StartSagaBuilder builder, Consumer<StartSagaBuilder> consumer) {
DefaultTransactionDefinition definition = new DefaultTransactionDefinition();
definition.setPropagationBehavior(DefaultTransactionDefinition.PROPAGATION_REQUIRED);
definition.setIsolationLevel(TransactionDefinition.ISOLATION_DEFAULT);
apply(builder, consumer, definition);
}
private void apply(StartSagaBuilder builder, Consumer<StartSagaBuilder> consumer,
TransactionDefinition definition) {
String id = generateUUID();
TransactionStatus status = transactionManager.getTransaction(definition);
builder.withUuid(id).withService(service);
try {
sagaClient.preConfirmSaga(builder.preBuild());
consumer.accept(builder);
transactionManager.commit(status);
} catch (Exception e) {
transactionManager.rollback(status);
sagaClient.cancelSaga(id);
}
sagaClient.confirmSaga(builder.confirmBuild());
}
private String generateUUID() {
return UUID.randomUUID().toString().replaceAll("-", "");
}
}
事务管理器收到事务发起者的信号后,会生成对应的事务任务,这些事务任务是可以按顺序执行也可以并发在同一个顺序执行的。
事务接收者这边会定时对事务管理器发起请求,看是否有事务任务,有事务任务的话找到对应的事务方法利用反射执行。这里是利用反射执行对应的事务方法的,那就需要事务接收者启动的时候将反射方法保存到本地。
启动时候将事务反射方法保持到本地,利用BeanPostProcessor
和反射
/**
* 将事务任务保持到本地
* @author Tarzan写bug
* @since 2022/11/05
*/
public class SagaTaskProcessor implements BeanPostProcessor {
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
Method[] methods = ReflectionUtils.getAllDeclaredMethods(bean.getClass());
for (Method method : methods) {
SagaTask sagaTask = AnnotationUtils.findAnnotation(method, SagaTask.class);
if (sagaTask != null) {
String key = sagaTask.code() + sagaTask.sagaCode();
/// TODO: 校验key唯一性
Method fallCallbackMethod = null;
Class<?> fallCallbackClazz = null;
if (!StringUtils.isEmpty(sagaTask.failCallbackMethod())) {
String clazzStr = sagaTask.failCallbackMethod();
String methodStr = sagaTask.failCallbackMethod();
try {
fallCallbackClazz = Class.forName(clazzStr);
fallCallbackMethod = fallCallbackClazz.getDeclaredMethod(methodStr, String.class);
fallCallbackMethod.setAccessible(true);
} catch (ClassNotFoundException | NoSuchMethodException e) {
e.printStackTrace();
}
}
SagaReceiver.invokeBeanMap.put(key, new SagaTaskInvokeBean(method, bean, fallCallbackClazz,
fallCallbackMethod, key, sagaTask));
}
}
return bean;
}
}
定时拉取事务任务,接收到事务任务后执行对应的反射方法
/**
* Saga事务接收者抽象类
* @author Tarzan写bug
* @since 2022/11/07
*/
public abstract class AbstractSagaReceiver {
private static final Logger LOGGER = LoggerFactory.getLogger(AbstractSagaReceiver.class);
protected String service;
protected String instance;
protected PlatformTransactionManager transactionManager;
public AbstractSagaReceiver(String service, String instance, long pollIntervalMs,
PlatformTransactionManager transactionManager,
ScheduledExecutorService scheduledExecutorService) {
this.service = service;
this.instance = instance;
this.transactionManager = transactionManager;
scheduledExecutorService.scheduleWithFixedDelay(() -> {
try {
scheduleRunning(instance);
} catch (Exception e) {
LOGGER.error("SagaReceiver schedule running error: {}", e.getMessage());
}
}, 20000, pollIntervalMs, TimeUnit.MILLISECONDS);
}
abstract void scheduleRunning(String instance);
}
/**
* saga接收者
* @author Tarzan写bug
* @since 2022/11/05
*/
public class SagaReceiver extends AbstractSagaReceiver {
public static final Map<String, SagaTaskInvokeBean> invokeBeanMap = new HashMap<>();
private PollSagaTaskInstanceDTO pollDTO;
private SagaProperties sagaProperties;
private SagaClient sagaClient;
public SagaReceiver(String service, String instance, long pollIntervalMs,
PlatformTransactionManager transactionManager,
ScheduledExecutorService scheduledExecutorService) {
super(service, instance, pollIntervalMs, transactionManager, scheduledExecutorService);
}
@Override
void scheduleRunning(String instance) {
List<SagaTaskInstanceDTO> sagaTasks = sagaClient.pollBatch(getPollDTO());
Optional.ofNullable(sagaTasks).ifPresent(list -> {
list.forEach(item -> {
SagaTaskInvokeBean invokeBean = invokeBeanMap.get(item.getSagaCode() + item.getTaskCode());
TransactionStatus transactionStatus = createTransactionStatus(transactionManager);
try {
invokeBean.getMethod().setAccessible(true);
Object result = invokeBean.getMethod().invoke(invokeBean.getObject(), item.getInput());
sagaClient.updateStatus();
transactionManager.commit(transactionStatus);
} catch (IllegalAccessException e) {
transactionManager.rollback(transactionStatus);
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
});
});
}
private PollSagaTaskInstanceDTO getPollDTO() {
if (pollDTO == null) {
pollDTO = new PollSagaTaskInstanceDTO(service, instance, sagaProperties.getPollIntervalMs());
}
return pollDTO;
}
private TransactionStatus createTransactionStatus(PlatformTransactionManager transactionManager) {
DefaultTransactionDefinition definition = new DefaultTransactionDefinition();
definition.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
definition.setIsolationLevel(TransactionDefinition.ISOLATION_REPEATABLE_READ);
return transactionManager.getTransaction(definition);
}
}
Saga理论里描述了两种事务失败后的操作,分别是向前恢复和向后恢复;
向前恢复:即失败的事务重试,直到成功为止,也就是整个事务流程最终必须成功;
向后恢复:当有事务流程为T1→T2→T3,执行到T3失败后,根据设置后的回滚方法按C3→C2→C1这样的顺序回滚。
上面是对Saga
分布式事务的一种实现思路,并没有完整的实现代码,后期有空再完善地补充。思路代码收录在https://gitee.com/ouwenrts/tuyere.git
.
世界那么大,感谢遇见,未来可期…
欢迎同频共振的那一部分人
作者公众号:Tarzan写bug