Tx-lcn lcn-client 客户端执行逻辑（6）
一、源码分析入口@EnableDistributedTransaction
客户端只需在启动类上 增加@EnableDistributedTransaction 即可实现 分布式事务，所以我们以此为切入点。
我们从注解@EnableDistributedTransaction开始，这个注解是开启分布式事务客户端的唯一注解。
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
@Documented
@Import({TCAutoConfiguration.class, DependenciesImportSelector.class})
public @interface EnableDistributedTransaction {
boolean enableTxc() default true;
}
跟进TCAutoConfiguration 源码：
@Configuration
@ComponentScan(
excludeFilters = @ComponentScan.Filter(
type = FilterType.ASPECTJ, pattern = "com.codingapi.txlcn.tc.core.transaction.txc..*"
)
)
@Import({TxLoggerConfiguration.class, TracingAutoConfiguration.class})
public class TCAutoConfiguration {
/**
* All initialization about TX-LCN
*
* @param applicationContext Spring ApplicationContext
* @return TX-LCN custom runner
*/
@Bean
public ApplicationRunner txLcnApplicationRunner(ApplicationContext applicationContext) {
return new TxLcnApplicationRunner(applicationContext);
}
@Bean
@ConditionalOnMissingBean
public ModIdProvider modIdProvider(ConfigurableEnvironment environment,
@Autowired(required = false) ServerProperties serverProperties) {
return () -> ApplicationInformation.modId(environment, serverProperties);
}
}
代码比服务端少，从注释上看所有的功能都在构建ApplicationRunner上，主要是默认执行其run（）方法。
@Override
public void run(ApplicationArguments args) throws Exception {
//找到bean 实现了TxLcnInitializer接口的bean
Map runnerMap = applicationContext.getBeansOfType(TxLcnInitializer.class);
// 根据实现了Order 接口 对已经实现了TxLcnInitializer接口的bean 进行排序
initializers = runnerMap.values().stream().sorted(Comparator.comparing(TxLcnInitializer::order))
.collect(Collectors.toList());
// 遍历initializers 对每个txLcnInitializer 执行init 方法
for (TxLcnInitializer txLcnInitializer : initializers) {
txLcnInitializer.init();
}
} 代码和服务端是一样的，也是找到所有的TxLcnInitializer,然后调用其init方法。
二、分解各种实现了TxLcnInitializer接口的Bean的init
2.1 DTXCheckingInitialization分布式事务检测初始化器
@Component
public class DTXCheckingInitialization implements TxLcnInitializer {
private final DTXChecking dtxChecking;
private final TransactionCleanTemplate transactionCleanTemplate;
@Autowired
public DTXCheckingInitialization(DTXChecking dtxChecking, TransactionCleanTemplate transactionCleanTemplate) {
this.dtxChecking = dtxChecking;
this.transactionCleanTemplate = transactionCleanTemplate;
}
@Override
public void init() throws Exception {
if (dtxChecking instanceof SimpleDTXChecking) {
((SimpleDTXChecking) dtxChecking).setTransactionCleanTemplate(transactionCleanTemplate);
}
}
}代码很简单，该类持有两个对象，分布式事务检测器与事务清理模板，init根据DTXChecking的类型设置了事务清理模板。
2.2 TCRpcServer客户端
@Component
public class TCRpcServer implements TxLcnInitializer {
private final RpcClientInitializer rpcClientInitializer;
private final TxClientConfig txClientConfig;
private final RpcConfig rpcConfig;
@Autowired
public TCRpcServer(RpcClientInitializer rpcClientInitializer,
TxClientConfig txClientConfig, RpcConfig rpcConfig) {
this.rpcClientInitializer = rpcClientInitializer;
this.txClientConfig = txClientConfig;
this.rpcConfig = rpcConfig;
}
@Override
public void init() throws Exception {
// rpc timeout (ms)
if (rpcConfig.getWaitTime() <= 5) {
rpcConfig.setWaitTime(1000);
}
// rpc client init.
rpcClientInitializer.init(TxManagerHost.parserList(txClientConfig.getManagerAddress()), false);
}
}
2.3 NettyRpcClientInitializer
@Component
@Slf4j
public class NettyRpcClientInitializer implements RpcClientInitializer, DisposableBean {
private static NettyRpcClientInitializer INSTANCE;
private final NettyRpcClientChannelInitializer nettyRpcClientChannelInitializer;
private final RpcConfig rpcConfig;
private final ClientInitCallBack clientInitCallBack;
private EventLoopGroup workerGroup;
@Autowired
public NettyRpcClientInitializer(NettyRpcClientChannelInitializer nettyRpcClientChannelInitializer, RpcConfig rpcConfig, ClientInitCallBack clientInitCallBack) {
this.nettyRpcClientChannelInitializer = nettyRpcClientChannelInitializer;
this.rpcConfig = rpcConfig;
this.clientInitCallBack = clientInitCallBack;
INSTANCE = this;
}
public static void reConnect(SocketAddress socketAddress) {
Objects.requireNonNull(socketAddress, "non support!");
INSTANCE.connect(socketAddress);
}
@Override
public void init(List hosts, boolean sync) {
NettyContext.type = NettyType.client;
NettyContext.params = hosts;
workerGroup = new NioEventLoopGroup();
for (TxManagerHost host : hosts) {
Optional future = connect(new InetSocketAddress(host.getHost(), host.getPort()));
if (sync && future.isPresent()) {
try {
future.get().get(10, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
log.error(e.getMessage(), e);
}
}
}
}
@Override
public synchronized Optional connect(SocketAddress socketAddress) {
for (int i = 0; i < rpcConfig.getReconnectCount(); i++) {
if (SocketManager.getInstance().noConnect(socketAddress)) {
try {
log.info("Try connect socket({}) - count {}", socketAddress, i + 1);
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 5000);
b.handler(nettyRpcClientChannelInitializer);
return Optional.of(b.connect(socketAddress).syncUninterruptibly());
} catch (Exception e) {
log.warn("Connect socket({}) fail. {}ms latter try again.", socketAddress, rpcConfig.getReconnectDelay());
try {
Thread.sleep(rpcConfig.getReconnectDelay());
} catch (InterruptedException e1) {
e1.printStackTrace();
}
continue;
}
}
// 忽略已连接的连接
return Optional.empty();
}
log.warn("Finally, netty connection fail , socket is {}", socketAddress);
clientInitCallBack.connectFail(socketAddress.toString());
return Optional.empty();
}
@Override
public void destroy() {
workerGroup.shutdownGracefully();
log.info("RPC client was down.");
}
}
这里启动了一个netty客户端，根据manager-address配置连接到了服务器端。
上面的connect方法还实现了一个功能就是重连机制，根据配置的重连次数ReconnectCount（默认8）进行重连。默认重试8次，间隔6秒。
2.4 NettyRpcClientChannelInitializer
NettyRpcClientChannelInitializer实现了ChannelInitializer在启动客户端调用initChannel方法。
@Component
public class NettyRpcClientChannelInitializer extends ChannelInitializer {
@Autowired
private RpcAnswerHandler rpcAnswerHandler;
@Autowired
private NettyClientRetryHandler nettyClientRetryHandler;
@Autowired
private SocketManagerInitHandler socketManagerInitHandler;
@Autowired
private RpcCmdDecoder rpcCmdDecoder;
@Override
protected void initChannel(Channel ch) throws Exception {
ch.pipeline().addLast(new LengthFieldPrepender(4, false));
ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE,
0, 4, 0, 4));
ch.pipeline().addLast(new ObjectSerializerEncoder());
ch.pipeline().addLast(new ObjectSerializerDecoder());
ch.pipeline().addLast(rpcCmdDecoder);
ch.pipeline().addLast(new RpcCmdEncoder());
ch.pipeline().addLast(nettyClientRetryHandler);
ch.pipeline().addLast(socketManagerInitHandler);
//同服务端，但是功能少了一个功能
ch.pipeline().addLast(rpcAnswerHandler);
}
}
与服务端相比少了一个IdleStateHandler用于心跳检测，所以socketManagerInitHandler中少了一个功能即userEventTriggered不会被调用。
多了一个nettyClientRetryHandler,主要有两个作用
1、重连机制，默认8次间隔6秒。
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
super.channelInactive(ctx);
log.error("keepSize:{},nowSize:{}", keepSize, SocketManager.getInstance().currentSize());
SocketAddress socketAddress = ctx.channel().remoteAddress();
log.error("socketAddress:{} ", socketAddress);
//断线重连
NettyRpcClientInitializer.reConnect(socketAddress);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
log.error("NettyClientRetryHandler - exception . ", cause);
if (cause instanceof ConnectException) {
int size = SocketManager.getInstance().currentSize();
Thread.sleep(1000 * 15);
log.error("current size:{} ", size);
log.error("try connect tx-manager:{} ", ctx.channel().remoteAddress());
//断线重连
NettyRpcClientInitializer.reConnect(ctx.channel().remoteAddress());
}
//发送数据包检测是否断开连接.
ctx.writeAndFlush(heartCmd);
}public static void reConnect(SocketAddress socketAddress) {
Objects.requireNonNull(socketAddress, "non support!");
INSTANCE.connect(socketAddress);
} public synchronized Optional connect(SocketAddress socketAddress) {
for (int i = 0; i < rpcConfig.getReconnectCount(); i++) {
if (SocketManager.getInstance().noConnect(socketAddress)) {
try {
log.info("Try connect socket({}) - count {}", socketAddress, i + 1);
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.option(ChannelOption.SO_KEEPALIVE, true);
b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 5000);
b.handler(nettyRpcClientChannelInitializer);
return Optional.of(b.connect(socketAddress).syncUninterruptibly());
} catch (Exception e) {
log.warn("Connect socket({}) fail. {}ms latter try again.", socketAddress, rpcConfig.getReconnectDelay());
try {
Thread.sleep(rpcConfig.getReconnectDelay());
} catch (InterruptedException e1) {
e1.printStackTrace();
}
continue;
}
}
// 忽略已连接的连接
return Optional.empty();
}
log.warn("Finally, netty connection fail , socket is {}", socketAddress);
clientInitCallBack.connectFail(socketAddress.toString());
return Optional.empty();
} 当链接断开或者发生异常时会进行重连机制，可以看到这里就是调用了connet方法进行重连的。
2、连接成功后回调机制。
回调作用
2.1、从服务端获取机器id、分布式事物超时时间、最大等待时间等参数（客户端不能配置这些参数需要以服务端为准）
2.2、如果服务端启动的数量大于客户端配置的服务器数量，会通过回调使得客户端连接所有的服务端。
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
keepSize = NettyContext.currentParam(List.class).size();
//回调函数
clientInitCallBack.connected(ctx.channel().remoteAddress().toString());
}public void connected(String remoteKey) {
//监听，在连接成功执行，此处为空实现
rpcEnvStatusListeners.forEach(rpcEnvStatusListener -> rpcEnvStatusListener.onConnected(remoteKey));
new Thread(() -> {
try {
log.info("Send init message to TM[{}]", remoteKey);
//向服务端发送消息，获取配置信息
MessageDto msg = rpcClient.request(
remoteKey, MessageCreator.initClient(applicationName, modIdProvider.modId()), 5000);
if (MessageUtils.statusOk(msg)) {
//每一次建立连接时将会获取最新的时间
InitClientParams resParams = msg.loadBean(InitClientParams.class);
// 1. 设置DTX Time 、 TM RPC timeout 和 MachineId
txClientConfig.applyDtxTime(resParams.getDtxTime());
txClientConfig.applyTmRpcTimeout(resParams.getTmRpcTimeout());
txClientConfig.applyMachineId(resParams.getMachineId());
// 2. IdGen 初始化
IdGenInit.applyDefaultIdGen(resParams.getSeqLen(), resParams.getMachineId());
// 3. 日志
log.info("Finally, determined dtx time is {}ms, tm rpc timeout is {} ms, machineId is {}",
resParams.getDtxTime(), resParams.getTmRpcTimeout(), resParams.getMachineId());
// 4. 执行其它监听器
rpcEnvStatusListeners.forEach(rpcEnvStatusListener -> rpcEnvStatusListener.onInitialized(remoteKey));
return;
}
log.error("TM[{}] exception. connect fail!", remoteKey);
} catch (RpcException e) {
log.error("Send init message exception: {}. connect fail!", e.getMessage());
}
}).start();
}1、向服务端发送消息，获取服务端配置
MessageDto msg = rpcClient.request(
remoteKey, MessageCreator.initClient(applicationName, modIdProvider.modId()), 5000);//构造消息体，设置action为init
public static MessageDto initClient(String appName, String labelName) {
InitClientParams initClientParams = new InitClientParams();
initClientParams.setAppName(appName);
initClientParams.setLabelName(labelName);
MessageDto messageDto = new MessageDto();
messageDto.setData(initClientParams);
messageDto.setAction(MessageConstants.ACTION_INIT_CLIENT);
return messageDto;
}//构造发送消息，发送
public MessageDto request(String remoteKey, MessageDto msg, long timeout) throws RpcException {
long startTime = System.currentTimeMillis();
NettyRpcCmd rpcCmd = new NettyRpcCmd();
rpcCmd.setMsg(msg);
String key = rpcCmd.randomKey();
rpcCmd.setKey(key);
rpcCmd.setRemoteKey(remoteKey);
MessageDto result = request0(rpcCmd, timeout);
log.debug("cmd request used time: {} ms", System.currentTimeMillis() - startTime);
return result;
}服务端是通过InitClientService类去处理消息的
public Serializable execute(TransactionCmd transactionCmd) throws TxManagerException {
//获取参数
InitClientParams initClientParams = transactionCmd.getMsg().loadBean(InitClientParams.class);
log.info("Registered TC: {}", initClientParams.getLabelName());
try {
//绑定
rpcClient.bindAppName(transactionCmd.getRemoteKey(), initClientParams.getAppName(), initClientParams.getLabelName());
} catch (RpcException e) {
throw new TxManagerException(e);
}
//以下为把服务端的一些信息放到参数中返回到客户端
// Machine len and id
initClientParams.setSeqLen(txManagerConfig.getSeqLen());
//服务端生成机器id
initClientParams.setMachineId(managerService.machineIdSync());
// DTX Time and TM timeout.
initClientParams.setDtxTime(txManagerConfig.getDtxTime());
initClientParams.setTmRpcTimeout(rpcConfig.getWaitTime());
// TM Name
initClientParams.setAppName(modIdProvider.modId());
return initClientParams;
}绑定，构建AppInfo和remoteKey关联，存入appNames
public void bindAppName(String remoteKey, String appName,String labelName) throws RpcException {
SocketManager.getInstance().bindModuleName(remoteKey, appName,labelName);
}public void bindModuleName(String remoteKey, String appName,String labelName) throws RpcException{
AppInfo appInfo = new AppInfo();
appInfo.setAppName(appName);
appInfo.setLabelName(labelName);
appInfo.setCreateTime(new Date());
if(containsLabelName(labelName)){
throw new RpcException("labelName:"+labelName+" has exist.");
}
appNames.put(remoteKey, appInfo);
}2、把服务端返回的信息设置到本config
//每一次建立连接时将会获取最新的时间
InitClientParams resParams = msg.loadBean(InitClientParams.class);
// 1. 设置DTX Time 、 TM RPC timeout 和 MachineId
txClientConfig.applyDtxTime(resParams.getDtxTime());
txClientConfig.applyTmRpcTimeout(resParams.getTmRpcTimeout());
txClientConfig.applyMachineId(resParams.getMachineId());
// 2. IdGen 初始化
IdGenInit.applyDefaultIdGen(resParams.getSeqLen(), resParams.getMachineId());3、执行监听器的onInitialized方法
此处监听器只有AutoTMClusterEngine类，onInitialized方法用来搜索所有的服务端，使客户端与其连接
什么意思呢？
我们知道客户端配置连接服务端地址并不需要把所有的服务端地址都要写上，只需要写上一个或是几个就能使客户端都自动连接上没有配置上的服务端，这是怎么实现的呢，就在这个方法里。
我们举个例子：启动了两个服务端A,B,只有一个客户端C，C客户端只配置了连接A服务端，那么客户端C是怎么和服务端B自动连接上的呢？
@1、客户端C启动时会根据配置启动一个netty客户端去连接服务端A。
@2、客户端C完成连接初始化后，会触发channelActive时间，调用channelActive方法，进而调用connected，从服务端获取配置信息，并设置到本地
@3、最后会调用onInitialized方法，主要作用就是检测我的客户端C是否与所有的服务端都连接了。这里有两个参数，一个是配置的服务端地址size在本例中只配置了一个，所以size为1；另一个是尝试连接的数量tryConnectCount这个默认是0，每连接一次加1；如果size=tryConnectCount时，会去通过netty向服务端发送消息，寻找所有的TM数量，TM收到信息，查询redis中的tm.instances（前面讲过，存储的是所有已经启动的TM地址信息）值封装后返回客户端。客户端收到所有的服务端地址（A,B），然后排除已经连接的地址（A），会再启动一个netty客户端去连接B
总体流程就是上面那样子，但是一些细节需要在代码中体现。
public void onInitialized(String remoteKey) {
//准备寻找TM
if (prepareToResearchTMCluster()) {
TMSearcher.echoTmClusterSize();
}
} private AtomicInteger tryConnectCount = new AtomicInteger(0);
//返回值为true为所有结束
private boolean prepareToResearchTMCluster() {
//原子类值加1，每连接一次都会加上1
int count = tryConnectCount.incrementAndGet();
//客户端配置的服务端地址数量
int size = txClientConfig.getManagerAddress().size();
//三种情况不同的场景
if (count == size) {
TMSearcher.search();
return false;
} else if (count > size) {
return !TMSearcher.searchedOne();
}
return true;
}public static void search() {
Objects.requireNonNull(RPC_CLIENT_INITIALIZER);
log.info("Searching for more TM...");
try {
//获取服务端返回的TM信息
HashSet cluster = RELIABLE_MESSENGER.queryTMCluster();
if (cluster.isEmpty()) {
log.info("No more TM.");
echoTMClusterSuccessful();
return;
}
//CountDownLatch
clusterCountLatch = new CountDownLatch(cluster.size() - knownTMClusterSize);
log.debug("wait connect size is {}", cluster.size() - knownTMClusterSize);
//启动netty客户端完成连接服务端
RPC_CLIENT_INITIALIZER.init(TxManagerHost.parserList(new ArrayList<>(cluster)), true);
//阻塞等待
clusterCountLatch.await(10, TimeUnit.SECONDS);
echoTMClusterSuccessful();
} catch (RpcException | InterruptedException e) {
throw new IllegalStateException("There is no normal TM.");
}
} public static boolean searchedOne() {
if (Objects.nonNull(clusterCountLatch)) {
if (clusterCountLatch.getCount() == 0) {
return false;
}
//减一
clusterCountLatch.countDown();
return true;
}
return false;
}我们对所有的场景做一个分析
1、服务端数量与客户端配置数量相等，则会执行size>count 与count=size
2、客户端配置的数量小于服务端数量，三种情况都会执行
我们一第二种情况做分析，假定服务端4个A,B,C,D 客户端E配置两个A,B
首先客户端E启动两个netty客户端去连接A,B，并且执行回调启动两个线程1,2。
线程1首先执行prepareToResearchTMCluster方法，size=2,count=1 这时size>count 返回true 打印所有的TM
线程2执行prepareToResearchTMCluster方法，size=2，count=2 这时size=count 执行search方法，获取到服务端的所有TM为4个，CountDownLatch的值为2，并且又启动两个netty客户端去连接C,D,又启动两个回调线程3,4，这是线程2是阻塞的，阻塞完成打印所有的TM。
线程3执行prepareToResearchTMCluster方法，size=2，count=3 这时size 线程4执行prepareToResearchTMCluster方法，size=2，count=4 这时size