分布式- Mysql + Mongo多数据源的分布式事务回滚

1. XA两阶段提交模型

如上图，XA规范实现的两阶段提交流程：（下面全部翻译自XA规范原文）

阶段1：

　　TM要求所有RMs准备提交(或准备)事务分支。这询问RM是否能够保证提交事务分支的能力。RM可能会查询该RM内部的其他实例。CRM被要求准备它们创建的事务分支,将prepare请求发送到远程站点并接收结果。在返回失败并回滚其工作之后，RM可以丢弃事务分支的信息。

阶段2：

　　TM根据实际情况向所有RMs发出提交或回滚事务分支的请求。CRM被要求提交或回滚它们创建的事务分支,向远程站点发送提交或回滚请求并接收结果。所有RMs提交或回滚对共享资源的更改，然后将状态返回给TM。然后TM可以丢弃全局事务的信息。

1.1 XA对2PC的优化

• 只读断言

　　当事务分支没有更新共享资源时，这个RM会断言并响应给TM的prepare请求。也就免去了阶段2。但是，如果一个RM在全局事务的所有RMs返回prepared之前返回了只读优化，该RM释放事务上下文，例如read locks。这时候其他事务就有机会去改变这些数据（可能是写锁），显然全局序列化被破坏。同样CRM也可以断言，当TM挂起或终止线程与事务分支的关联时，它不是某个特定线程中活动的事务分支的参与者。

• 一阶段提交

　　如果一个TM知道DTP系统中只有一个RM在修改共享资源，那么它可以使用单阶段提交。即TM免去了阶段1的prepare，直接执行了阶段2的commit。

1.2 2PC的缺点

• 资源阻塞

由于协调者的重要性，一旦协调者TM发生故障。参与者RM会一直阻塞下去。尤其在第二阶段，协调者发生故障，那么所有的参与者还都处于锁定事务资源的状态中，而无法继续完成事务操作。（如果是协调者挂掉，可以重新选举一个协调者，但是无法解决因为协调者宕机导致的参与者处于阻塞状态的问题）

• 数据不一致

在阶段二，当协调者向参与者发送commit请求之后，发生了局部网络异常或者在发送commit请求过程中协调者发生了故障，导致只有一部分参与者接受到了commit请求。而在这部分参与者接到commit请求之后就会执行commit操作。但是其他部分未接到commit请求的机器则无法执行事务提交。于是整个分布式系统便出现了数据不一致性的现象。

由于二阶段提交存在着这些缺陷，所以，研究者们在二阶段提交的基础上做了改进，提出了三阶段提交(该篇文章暂不涉及)。

2. Spring集成jta-atomikos实现Mysql + Mongo多数据源事务回滚

（1）项目环境为SpringBoot2.1.6

（2）依赖版本管理

    
        std-boot-starter-transaction
        priv.whh.std
        1.0.0-SNAPSHOT
        ../pom.xml
    
    4.0.0

    std-boot-jta
    
        1.1.23
        1.3.2
        3.4.6
        1.3.2
        8.0.11
    

    
        
            org.springframework.boot
            spring-boot-starter-data-mongodb
        
        
            org.springframework.boot
            spring-boot-starter-jta-atomikos
        
        
            com.alibaba
            druid
        
        
            org.projectlombok
            lombok
        
        
            org.mybatis.spring.boot
            mybatis-spring-boot-starter
            test
        
        
            org.mybatis
            mybatis-spring
            test
        
        
            org.mybatis
            mybatis
            test
        
        
            org.springframework.boot
            spring-boot-starter-web
            test
        
        
            mysql
            mysql-connector-java
            test
        
        
            org.springframework
            spring-jdbc
            test
        
    

    
        
            
                com.alibaba
                druid
                ${druid.version}
            
            
                org.mybatis.spring.boot
                mybatis-spring-boot-starter
                ${mybatis-spring-boot-starter.version}
            
            
                org.mybatis
                mybatis-spring
                ${mybatis-spring.version}
            
            
                org.mybatis
                mybatis
                ${mybatis.version}
            
            
                mysql
                mysql-connector-java
                ${mysql-connector-java.version}
            
        
    

（3）配置类JtaAutoConfiguration

package priv.whh.std.boot.jta.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.mongodb.MongoDbFactory;
import org.springframework.data.mongodb.MongoTransactionManager;
import priv.whh.std.boot.jta.manager.JtaUserTransaction;
import priv.whh.std.boot.jta.util.MongoUtils;

/**
 * @author whh
 * @date 2020/7/22
 */
@Configuration
public class JtaAutoConfiguration {
    @Bean
    @Primary
    public JtaUserTransaction jtaUserTransaction(MongoDbFactory factory, MongoUtils mongoUtils) {
        return new JtaUserTransaction(new MongoTransactionManager(factory), mongoUtils);
    }
}

（4）自定义事务JtaUserTransaction

package priv.whh.std.boot.jta.manager;

import com.atomikos.icatch.config.UserTransactionService;
import com.atomikos.icatch.config.UserTransactionServiceImp;
import com.atomikos.icatch.jta.TransactionManagerImp;
import com.atomikos.util.SerializableObjectFactory;
import com.mongodb.MongoException;
import com.mongodb.TransactionOptions;
import com.mongodb.client.ClientSession;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.mongodb.MongoDatabaseUtils;
import org.springframework.data.mongodb.MongoDbFactory;
import org.springframework.data.mongodb.MongoTransactionManager;
import org.springframework.data.mongodb.SessionSynchronization;
import org.springframework.lang.Nullable;
import org.springframework.transaction.TransactionDefinition;
import org.springframework.transaction.TransactionSystemException;
import org.springframework.transaction.support.*;
import org.springframework.util.Assert;
import org.springframework.util.ClassUtils;
import priv.whh.std.boot.jta.util.MongoUtils;

import javax.naming.NamingException;
import javax.naming.Reference;
import javax.naming.Referenceable;
import javax.transaction.NotSupportedException;
import javax.transaction.SystemException;
import javax.transaction.TransactionManager;
import javax.transaction.UserTransaction;
import java.io.Serializable;
import java.lang.reflect.Method;
import java.util.Objects;

/**
 * @author whh
 * @date 2020/7/23
 */
@Slf4j
public class JtaUserTransaction implements UserTransaction, Serializable, Referenceable {
    private static final long serialVersionUID = -865418426269785202L;
    private transient TransactionManager transactionManager;
    private transient MongoTransactionManager mongoTransactionManager;
    private final transient MongoUtils mongoUtils;

    public JtaUserTransaction(MongoTransactionManager mongoTransactionManager, MongoUtils mongoUtils) {
        this.mongoTransactionManager = mongoTransactionManager;
        this.mongoUtils = mongoUtils;
    }

    /**
     * @see javax.transaction.UserTransaction
     */
    @Override
    public void begin() throws NotSupportedException, SystemException {
        checkSetup();
        transactionManager.begin();
        mongoUtils.setSessionSynchronizationForTransactionBegin();
    }

    /**
     * @see javax.transaction.UserTransaction
     */
    @Override
    public void commit() throws javax.transaction.RollbackException, javax.transaction.HeuristicMixedException,
        javax.transaction.HeuristicRollbackException, javax.transaction.SystemException {
        Assert.notNull(mongoTransactionManager.getDbFactory(), "DbFactory must not be null!");
        if (Objects.nonNull(TransactionSynchronizationManager.getResource(mongoTransactionManager.getDbFactory()))) {
            MongoTransactionObject mongoTransactionObject = extractMongoTransaction(getMongoTransaction());
            MongoResourceHolder resourceHolder = newResourceHolder(new DefaultTransactionDefinition());
            mongoTransactionObject.setResourceHolder(resourceHolder);
            try {
                mongoTransactionObject.commitTransaction();
                TransactionSynchronizationManager.unbindResource(mongoTransactionManager.getDbFactory());
                mongoTransactionObject.getRequiredResourceHolder().clear();
                mongoTransactionObject.closeSession();
                mongoUtils.setSessionSynchronizationForTransactionCompletion();
            } catch (Exception ex) {
                throw new TransactionSystemException(String.format("Could not commit Mongo transaction for session %s.",
                    debugString(mongoTransactionObject.getSession())), ex);
            }
        }
        checkSetup();
        transactionManager.commit();
    }

    /**
     * @see javax.transaction.UserTransaction
     */
    @Override
    public void rollback() throws SystemException {
        Assert.notNull(mongoTransactionManager.getDbFactory(), "Db factory must not be null");
        if (Objects.nonNull(TransactionSynchronizationManager.getResource(mongoTransactionManager.getDbFactory()))) {
            MongoTransactionObject mongoTransactionObject = extractMongoTransaction(getMongoTransaction());
            MongoResourceHolder resourceHolder = newResourceHolder(new DefaultTransactionDefinition());
            mongoTransactionObject.setResourceHolder(resourceHolder);
            try {
                mongoTransactionObject.abortTransaction();
                TransactionSynchronizationManager.unbindResource(mongoTransactionManager.getDbFactory());
                mongoTransactionObject.getRequiredResourceHolder().clear();
                mongoTransactionObject.closeSession();
                mongoUtils.setSessionSynchronizationForTransactionCompletion();
            } catch (MongoException ex) {
                throw new TransactionSystemException(String.format("Could not abort Mongo transaction for session %s.",
                    debugString(mongoTransactionObject.getSession())), ex);
            }
        }
        checkSetup();
        transactionManager.rollback();
    }

    /**
     * 不抛出异常进行回滚
     *
     * @see javax.transaction.UserTransaction
     */
    @Override
    public void setRollbackOnly() throws SystemException {
        checkSetup();
        transactionManager.setRollbackOnly();
    }

    /**
     * @see javax.transaction.UserTransaction
     */
    @Override
    public int getStatus() throws SystemException {
        checkSetup();
        return transactionManager.getStatus();
    }

    /**
     * @see javax.transaction.UserTransaction
     */
    @Override
    public void setTransactionTimeout(int seconds) throws SystemException {
        checkSetup();
        transactionManager.setTransactionTimeout(seconds);
    }

    /**
     * IMPLEMENTATION OF REFERENCEABLE
     */
    @Override
    public Reference getReference() throws NamingException {
        return SerializableObjectFactory.createReference(this);
    }

    protected int determineTimeout(TransactionDefinition definition) {
        if (definition.getTimeout() != TransactionDefinition.TIMEOUT_DEFAULT) {
            return definition.getTimeout();
        }
        return TransactionDefinition.TIMEOUT_DEFAULT;
    }

    /**
     * Referenceable mechanism requires later setup of transactionManager, otherwise binding
     * 

     * into JNDI already requires that TM is running.
     */
    private void checkSetup() {
        synchronized (TransactionManagerImp.class) {
            transactionManager = TransactionManagerImp.getTransactionManager();
            if (Objects.isNull(transactionManager)) {
                UserTransactionService uts = new UserTransactionServiceImp();
                uts.init();
                transactionManager = TransactionManagerImp.getTransactionManager();
            }
        }
    }

    private Object getMongoTransaction() {
        MongoDbFactory mongoDbFactory = mongoTransactionManager.getDbFactory();
        Assert.notNull(mongoDbFactory, "Db factory must not be null");
        MongoResourceHolder resourceHolder = (MongoResourceHolder) TransactionSynchronizationManager
            .getResource(mongoDbFactory);
        return new MongoTransactionObject(resourceHolder);
    }

    private static MongoTransactionObject extractMongoTransaction(Object transaction) {
        Assert.isInstanceOf(MongoTransactionObject.class, transaction,
            () -> String.format("Expected to find a %s but it turned out to be %s.", MongoTransactionObject.class,
                transaction.getClass()));
        return (MongoTransactionObject) transaction;
    }

    private MongoResourceHolder newResourceHolder(TransactionDefinition definition) {
        MongoDbFactory dbFactory = mongoTransactionManager.getDbFactory();
        Class mongoDatabaseUtilsClazz = MongoDatabaseUtils.class;
        ClientSession session = null;
        try {
            Method doGetSession = mongoDatabaseUtilsClazz.getDeclaredMethod(
                "doGetSession", MongoDbFactory.class, SessionSynchronization.class);
            doGetSession.setAccessible(true);
            session = (ClientSession) doGetSession.invoke(
                mongoDatabaseUtilsClazz.newInstance(), dbFactory, SessionSynchronization.ALWAYS);
        } catch (Exception e) {
            log.error("getSession err;", e.getCause());
        }
        if (Objects.nonNull(session)) {
            MongoResourceHolder resourceHolder = new MongoResourceHolder(session, dbFactory);
            resourceHolder.setTimeoutIfNotDefaulted(determineTimeout(definition));
            return resourceHolder;
        }
        return null;
    }

    private static String debugString(@Nullable ClientSession session) {
        if (session == null) {
            return "null";
        }
        String debugString = String.format("[%s@%s ", ClassUtils.getShortName(session.getClass()),
            Integer.toHexString(session.hashCode()));
        try {
            if (session.getServerSession() != null) {
                debugString += String.format("id = %s, ", session.getServerSession().getIdentifier());
                debugString += String.format("causallyConsistent = %s, ", session.isCausallyConsistent());
                debugString += String.format("txActive = %s, ", session.hasActiveTransaction());
                debugString += String.format("txNumber = %d, ", session.getServerSession().getTransactionNumber());
                debugString += String.format("closed = %s, ", session.getServerSession().isClosed());
                debugString += String.format("clusterTime = %s", session.getClusterTime());
            } else {
                debugString += "id = n/a";
                debugString += String.format("causallyConsistent = %s, ", session.isCausallyConsistent());
                debugString += String.format("txActive = %s, ", session.hasActiveTransaction());
                debugString += String.format("clusterTime = %s", session.getClusterTime());
            }
        } catch (RuntimeException e) {
            debugString += String.format("error = %s", e.getMessage());
        }
        debugString += "]";
        return debugString;
    }

    /**
     * @see org.springframework.data.mongodb.MongoResourceHolder
     */
    protected static class MongoTransactionObject implements SmartTransactionObject {
        @Nullable
        private MongoResourceHolder resourceHolder;

        MongoTransactionObject(@Nullable MongoResourceHolder resourceHolder) {
            this.resourceHolder = resourceHolder;
        }

        /**
         * Set the {@link MongoResourceHolder}.
         *
         * @param resourceHolder can be {@literal null}.
         */
        void setResourceHolder(@Nullable MongoResourceHolder resourceHolder) {
            this.resourceHolder = resourceHolder;
        }

        /**
         * @return {@literal true} if a {@link MongoResourceHolder} is set.
         */
        final boolean hasResourceHolder() {
            return resourceHolder != null;
        }

        /**
         * Start a MongoDB transaction optionally given {@link TransactionOptions}.
         *
         * @param options can be {@literal null}
         */

        void startTransaction(@Nullable TransactionOptions options) {
            ClientSession session = getRequiredSession();
            if (options != null) {
                session.startTransaction(options);
            } else {
                session.startTransaction();
            }
        }

        /**
         * Commit the transaction.
         */
        public void commitTransaction() {
            getRequiredSession().commitTransaction();
        }

        /**
         * Rollback (abort) the transaction.
         */
        public void abortTransaction() {
            getRequiredSession().abortTransaction();
        }

        /**
         * Close a {@link ClientSession} without regard to its transactional state.
         */
        void closeSession() {
            ClientSession session = getRequiredSession();
            if (session.getServerSession() != null && !session.getServerSession().isClosed()) {
                session.close();
            }
        }

        @Nullable
        public ClientSession getSession() {
            return resourceHolder != null ? resourceHolder.getSession() : null;
        }

        private MongoResourceHolder getRequiredResourceHolder() {
            Assert.state(resourceHolder != null, "MongoResourceHolder is required but not present. o_O");
            return resourceHolder;
        }

        private ClientSession getRequiredSession() {
            ClientSession session = getSession();
            Assert.state(session != null, "A Session is required but it turned out to be null.");
            return session;
        }

        /**
         * (non-Javadoc)
         *
         * @see org.springframework.transaction.support.SmartTransactionObject#isRollbackOnly()
         */
        @Override
        public boolean isRollbackOnly() {
            return this.resourceHolder != null && this.resourceHolder.isRollbackOnly();
        }

        /**
         * (non-Javadoc)
         *
         * @see org.springframework.transaction.support.SmartTransactionObject#flush()
         */
        @Override
        public void flush() {
            TransactionSynchronizationUtils.triggerFlush();
        }

    }

    /**
     * @see org.springframework.data.mongodb.MongoResourceHolder
     */
    class MongoResourceHolder extends ResourceHolderSupport {
        private @Nullable
        ClientSession session;
        private MongoDbFactory dbFactory;

        /**
         * Create a new {@link com.shero.sport.web.conf.JtaTransactionImp.MongoResourceHolder} for a given {@link ClientSession session}.
         *
         * @param session   the associated {@link ClientSession}. Can be {@literal null}.
         * @param dbFactory the associated {@link MongoDbFactory}. must not be {@literal null}.
         */
        MongoResourceHolder(@Nullable ClientSession session, MongoDbFactory dbFactory) {
            this.session = session;
            this.dbFactory = dbFactory;
        }

        /**
         * @return the associated {@link ClientSession}. Can be {@literal null}.
         */
        @Nullable
        ClientSession getSession() {
            return session;
        }

        /**
         * @return the required associated {@link ClientSession}.
         * @throws IllegalStateException if no {@link ClientSession} is associated with this {@link com.shero.sport.web.conf.JtaTransactionImp.MongoResourceHolder}.
         * @since 2.1.3
         */
        ClientSession getRequiredSession() {
            ClientSession session = getSession();
            if (session == null) {
                throw new IllegalStateException("No session available!");
            }
            return session;
        }

        /**
         * @return the associated {@link MongoDbFactory}.
         */
        public MongoDbFactory getDbFactory() {
            return dbFactory;
        }

        /**
         * Set the {@link ClientSession} to guard.
         *
         * @param session can be {@literal null}.
         */
        public void setSession(@Nullable ClientSession session) {
            this.session = session;
        }

        /**
         * Only set the timeout if it does not match the {@link TransactionDefinition#TIMEOUT_DEFAULT default timeout}.
         *
         * @param seconds
         */
        void setTimeoutIfNotDefaulted(int seconds) {
            if (seconds != TransactionDefinition.TIMEOUT_DEFAULT) {
                setTimeoutInSeconds(seconds);
            }
        }

        /**
         * @return {@literal true} if session is not {@literal null}.
         */
        boolean hasSession() {
            return session != null;
        }

        /**
         * @return {@literal true} if the session is active and has not been closed.
         */
        boolean hasActiveSession() {
            if (!hasSession()) {
                return false;
            }
            return hasServerSession() && !getRequiredSession().getServerSession().isClosed();
        }

        /**
         * @return {@literal true} if the session has an active transaction.
         * @see #hasActiveSession()
         * @since 2.1.3
         */
        boolean hasActiveTransaction() {
            if (!hasActiveSession()) {
                return false;
            }
            return getRequiredSession().hasActiveTransaction();
        }

        /**
         * @return {@literal true} if the {@link ClientSession} has a {@link com.mongodb.session.ServerSession} associated
         * 

         * that is accessible via {@link ClientSession#getServerSession()}.
         */
        boolean hasServerSession() {
            try {
                return getRequiredSession().getServerSession() != null;
            } catch (IllegalStateException serverSessionClosed) {
                // ignore
            }
            return false;
        }
    }
}

（5）工具类MongoUtils

package priv.whh.std.boot.jta.util;

import lombok.RequiredArgsConstructor;
import org.springframework.data.mongodb.SessionSynchronization;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.stereotype.Component;

/**
 * @author whh
 * @date 2020/7/22
 */
@Component
@RequiredArgsConstructor
public class MongoUtils {
    private final MongoTemplate mongoTemplate;

    public void setSessionSynchronizationForTransactionBegin() {
        // 同步任何事务（即使是空事务）并在执行此操作时启动MongoDB事务
        mongoTemplate.setSessionSynchronization(SessionSynchronization.ALWAYS);
    }

    public void setSessionSynchronizationForTransactionCompletion() {
        mongoTemplate.setSessionSynchronization(SessionSynchronization.ON_ACTUAL_TRANSACTION);
    }
}

（6）配置

mybatis:
  configuration:
    map-underscore-to-camel-case: true
  mapper-locations: classpath*:mapper/*.xml
  type-aliases-package: com.tzc.whh.model
spring:
  datasource:
    url: jdbc:mysql://127.0.0.1:3306/local?useUnicode=true&characterEncoding=UTF8&useSSL=false&serverTimezone=Asia/Shanghai
    username: root
    password: 123456
    #    driver-class-name: com.mysql.jdbc.Driver
    driver-class-name: com.mysql.cj.jdbc.Driver
    xa:
      data-source-class-name: com.alibaba.druid.pool.xa.DruidXADataSource
  data:
    mongodb:
      uri: mongodb://127.0.0.1:27018,127.0.0.1:27019/coupon?authSource=coupon&slaveOk=true&replicaSet=rs0&write=1&readPreference=secondaryPreferred&connectTimeoutMS=300000
server:
  port: 9090

（7）事务类Manager

package priv.whh.std.boot.jta.manager;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;
import priv.whh.std.boot.jta.dao.UserMapper;
import priv.whh.std.boot.jta.po.UserPo;

import java.util.Objects;
import java.util.Random;

/**
 * @author whh
 * @date 2020/7/22
 */
@Component
public class Manager {
    @Autowired
    private UserMapper userMapper;
    @Autowired
    private MongoTemplate mongoTemplate;

    @Transactional(rollbackFor = Exception.class)
    public void test(Integer test) throws Exception {
        userMapper.insert(new UserPo(1L, "test"));
        mongoTemplate.save(new UserPo(new Random().nextLong(), "testA"), "t_account");
        userMapper.insert(new UserPo(1L, "testB"));
        mongoTemplate.save(new UserPo(new Random().nextLong(), "testB"), "t_account");
        if (Objects.equals(1, test)) {
            throw new Exception();
        }
    }
}

当入参test为1时，抛出异常， 事务回滚。其它情况时，正常执行。

 

参考资料：

【分布式事务（一）原理概览】

【springboot + jta + mysql + mongo 分布式（多种数据源）事务】