什么是链路?
简单又狭义的理解,就是用户的一次操作,我们的程序可以将后端所有的操作记录全部串联起来。
在单体架构下,我们可以将controller层,service层,dao层的日志进行串联起来,当我们要排查问题时,只需要根据traceId就可以直接将这次操作的所有日志查询出来。同理,在分布式的架构中,一次操作涉及到n个服务,如查我们要想查询用户这次请求的所有记录查询出来,如果这些记录没有串联起来,试想要在多个服务日志去排查是多么困难。分布式的链路监控是一个大话题,里面还有spanId等,这个不在本文探讨中。
http及rpc拦截与日志实现
1.http我们可以使用Filter过滤器进行拦截
2.rpc如果采用的是dubbo框架,那么以采用对应的dubbo Filter过滤器进行拦截处理
注意:
原理都比较简单,网上资料很多。但是这里要注意的是,当你的业务使用了异步线程进行处理时,你的链路数据会丢失。一般传递traceId的方式是ThreadLocal,ThreadLocal不支持父子线程传递。此时我们可以使用jdk提供的InheritableThreaLocal来进行父子线程传递,但是这个也有对应的缺点,就是在线程池使用的时候不好使。所以通常最好的方使是使用阿里开源的transmittablethreadlocal来进行traceId的传递,切记使用完一定要记得将traceId clear掉。
日志
配置方式:
将自定义的链路 『trace_id』配置到 logback中(这里只配置控制台的方式,file文件配置方式一样)
%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{trace_id}] -[%thread] %-5level %logger{50}-%msg%n
我们模拟一个简单的请求代码:
@Slf4j
@RestController
public class TestController {
@GetMapping("/test-single")
public String testSingle() {
String traceId = UUID.randomUUID().toString();
MDC.put("trace_id", traceId);
log.info("----main test--");
handleBiz();
return "error";
}
private void handleBiz() {
log.info("----sub test--");
}
}
然后得到正常的结果
2022-09-24 12:18:39.135 [26a63d09-9501-4d03-a61f-e09abf79dc7b] -[http-nio-1111-exec-1] INFO com.lyqf.controller.TestController-----main test--
2022-09-24 12:18:39.135 [26a63d09-9501-4d03-a61f-e09abf79dc7b] -[http-nio-1111-exec-1] INFO com.lyqf.controller.TestController-----sub test--
然而事事并非总是这么的如愿,如果我们的业务使用了异步线程,就不好使了,我们把代码改一下:
@Slf4j
@RestController
public class TestController {
@GetMapping("/test-single")
public String testSingle() {
String traceId = UUID.randomUUID().toString();
MDC.put("trace_id", traceId);
log.info("----main test--");
new Thread(() -> {
handleBiz();
}).start();
return "error";
}
private void handleBiz() {
log.info("----sub test--");
}
}
2022-09-24 12:20:28.831 [07d0f883-0808-4b71-8c7d-356936a05ed8] -[http-nio-1111-exec-1] INFO com.lyqf.controller.TestController-----main test--
2022-09-24 12:20:28.831 [] -[Thread-208] INFO com.lyqf.controller.TestController-----sub test--
导致出现这个原因是logback里trace_id是获通过MDC来获取的,而MDC的实现方式是使用的ThreadLocal来实现的,所以在异步线程处理的时候获取到的traceId是空值。
总结一下:java异步通常用三种方式
1.直接创建单线程场景
2.使用CompletFuture及自定义线程场景
3.使用sping 及@Async 场景
后面我们将为每一种方式提供单独的解决方案。
解决方案
1.直接创建单线程场景
我们可以将traceId放在inheritableThreadLocal中,然后在异步线程中获取该值并放入MDC中从而实现。
@Slf4j
@RestController
public class TestController {
private InheritableThreadLocal inheritableThreadLocal = new InheritableThreadLocal<>();
@GetMapping("/test-single")
public String testSingle() {
String traceId = UUID.randomUUID().toString();
MDC.put("trace_id", traceId);
inheritableThreadLocal.set(traceId);
log.info("----main test--");
new Thread(() -> {
try {
MDC.put("trace_id", inheritableThreadLocal.get());
log.info("----sub test--");
} finally {
MDC.clear();
}
}).start();
inheritableThreadLocal.remove();
return "error";
}
}
执行结果
2022-09-23 17:27:00.038 [a9579bd9-c205-486a-a308-ab1c308d409b] -[http-nio-1111-exec-1] INFO com.lyqf.controller.TestController-----main test--
2022-09-23 17:27:00.038 [a9579bd9-c205-486a-a308-ab1c308d409b] -[Thread-205] INFO com.lyqf.controller.TestController-----sub test--
2. 使用CompletFuture及自定义线程场景
封装runnable、callable、Consumer任务类,为线程池提供使用。
public class MDCWrappers {
public static Runnable wrap(final Runnable runnable) {
final Map context = MDC.getCopyOfContextMap();
return () -> {
Map previous = MDC.getCopyOfContextMap();
if (context == null) {
MDC.clear();
} else {
MDC.setContextMap(context);
}
try {
runnable.run();
} finally {
if (previous == null) {
MDC.clear();
} else {
MDC.setContextMap(previous);
}
}
};
}
public static Callable wrap(final Callable callable) {
final Map context = MDC.getCopyOfContextMap();
return () -> {
Map previous = MDC.getCopyOfContextMap();
if (context == null) {
MDC.clear();
} else {
MDC.setContextMap(context);
}
try {
return callable.call();
} finally {
if (previous == null) {
MDC.clear();
} else {
MDC.setContextMap(previous);
}
}
};
}
public static Consumer wrap(final Consumer consumer) {
final Map context = MDC.getCopyOfContextMap();
return (t) -> {
Map previous = MDC.getCopyOfContextMap();
if (context == null) {
MDC.clear();
} else {
MDC.setContextMap(context);
}
try {
consumer.accept(t);
} finally {
if (previous == null) {
MDC.clear();
} else {
MDC.setContextMap(previous);
}
}
};
}
public static Collection> wrapCollection(Collection extends Callable> tasks) {
Collection> wrapped = new ArrayList<>();
for (Callable task : tasks) {
wrapped.add(wrap(task));
}
return wrapped;
}
}
自定义MDCThreadPoolExecutor且实现ExecutorService接口
public class MDCThreadPoolExecutor implements ExecutorService {
protected ExecutorService executorService;
MDCThreadPoolExecutor() {
}
public MDCThreadPoolExecutor(
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue) {
this.executorService = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
}
public MDCThreadPoolExecutor(
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadFactory threadFactory) {
this.executorService = new ThreadPoolExecutor(
corePoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
threadFactory);
}
public MDCThreadPoolExecutor(
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
RejectedExecutionHandler handler) {
this.executorService = new ThreadPoolExecutor(
corePoolSize,
maximumPoolSize,
keepAliveTime,
unit,
workQueue,
handler);
}
@Override
public Future submit(Callable task) {
return executorService.submit(MDCWrappers.wrap(task));
}
@Override
public Future submit(Runnable task, T result) {
return executorService.submit(MDCWrappers.wrap(task), result);
}
@Override
public Future> submit(Runnable task) {
return executorService.submit(MDCWrappers.wrap(task));
}
@Override
public List> invokeAll(Collection extends Callable> tasks) throws InterruptedException {
return executorService.invokeAll(MDCWrappers.wrapCollection(tasks));
}
@Override
public List> invokeAll(Collection extends Callable> tasks, long timeout, TimeUnit unit)
throws InterruptedException {
return executorService.invokeAll(MDCWrappers.wrapCollection(tasks), timeout, unit);
}
@Override
public T invokeAny(Collection extends Callable> tasks) throws InterruptedException, ExecutionException {
return executorService.invokeAny(MDCWrappers.wrapCollection(tasks));
}
@Override
public T invokeAny(Collection extends Callable> tasks, long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException, TimeoutException {
return executorService.invokeAny(MDCWrappers.wrapCollection(tasks), timeout, unit);
}
@Override
public void execute(Runnable command) {
executorService.execute(MDCWrappers.wrap(command));
}
@Override
public void shutdown() {
executorService.shutdown();
}
@Override
public List shutdownNow() {
return executorService.shutdownNow();
}
@Override
public boolean isShutdown() {
return executorService.isShutdown();
}
@Override
public boolean isTerminated() {
return executorService.isTerminated();
}
@Override
public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {
return executorService.awaitTermination(timeout, unit);
}
}
请求代码块
@Slf4j
@RestController
public class TestController {
MDCThreadPoolExecutor threadPoolExecutor =
new MDCThreadPoolExecutor(1, 1, 1, TimeUnit.HOURS, new ArrayBlockingQueue<>(10));
MDCThreadPoolExecutor completableFutureThreadPool =
new MDCThreadPoolExecutor(1, 1, 1, TimeUnit.HOURS, new ArrayBlockingQueue<>(10));
@GetMapping("/test-thread-pool")
public String testThreadPool() {
MDC.put("trace_id", UUID.randomUUID().toString());
log.info("----main test--");
threadPoolExecutor.execute(() -> {
log.info("----sub threadPoolExecutor test--");
});
CompletableFuture.supplyAsync(() -> {
log.info("----sub CompletableFuture test--");
return "success";
}, completableFutureThreadPool);
return "success";
}
}
输出结果:
2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[http-nio-1111-exec-2] INFO com.lyqf.controller.TestController-----main test--
2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[pool-5-thread-1] INFO com.lyqf.controller.TestController-----sub CompletableFuture test--
2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[pool-4-thread-1] INFO com.lyqf.controller.TestController-----sub threadPoolExecutor test--
3.使用sping 及@Async 场景
public class MdcTaskDecorator implements TaskDecorator {
@Override
public Runnable decorate(Runnable runnable) {
Map contextMap = MDC.getCopyOfContextMap();
return () -> {
try {
MDC.setContextMap(contextMap);
runnable.run();
} finally {
MDC.clear();
}
};
}
}
配置Bean
@Configuration
public class MdcConfig {
@Bean
public AsyncTaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
// 设置线程名称
executor.setThreadNamePrefix("customer-executor");
// 设置最大线程数
executor.setMaxPoolSize(50);
// 设置核心线程数
executor.setCorePoolSize(10);
// 设置线程空闲时间,默认60
executor.setKeepAliveSeconds(60);
// 设置队列容量
executor.setQueueCapacity(1000);
executor.setTaskDecorator(new MdcTaskDecorator());
// 设置拒绝策略
executor.setRejectedExecutionHandler(new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
//todo
}
});
return executor;
}
}
构建请求controller
@Slf4j
@RestController
public class TestController {
@Resource
private TestService testService;
@GetMapping("/test")
public String test() {
MDC.put("trace_id", UUID.randomUUID().toString());
log.info("----main test--");
testService.test();
return "success";
}
异步代码块
@Slf4j
@Service
public class TestService {
@Async
public void test() {
log.info("----TestService test--");
}
}
输出结果
2022-09-23 17:11:12.190 [311dd086d94246e4b819c79bec72350f] -[http-nio-1111-exec-2] INFO com.lyqf.controller.TestController-----main test--
2022-09-23 17:11:12.191 [311dd086d94246e4b819c79bec72350f] -[customer-executor2] INFO com.lyqf.TestService-----TestService test--