聊一聊链路日志-异步线程

什么是链路?

简单又狭义的理解,就是用户的一次操作,我们的程序可以将后端所有的操作记录全部串联起来。

在单体架构下,我们可以将controller层,service层,dao层的日志进行串联起来,当我们要排查问题时,只需要根据traceId就可以直接将这次操作的所有日志查询出来。同理,在分布式的架构中,一次操作涉及到n个服务,如查我们要想查询用户这次请求的所有记录查询出来,如果这些记录没有串联起来,试想要在多个服务日志去排查是多么困难。分布式的链路监控是一个大话题,里面还有spanId等,这个不在本文探讨中。

http及rpc拦截与日志实现

1.http我们可以使用Filter过滤器进行拦截
2.rpc如果采用的是dubbo框架,那么以采用对应的dubbo Filter过滤器进行拦截处理
注意:
原理都比较简单,网上资料很多。但是这里要注意的是,当你的业务使用了异步线程进行处理时,你的链路数据会丢失。一般传递traceId的方式是ThreadLocal,ThreadLocal不支持父子线程传递。此时我们可以使用jdk提供的InheritableThreaLocal来进行父子线程传递,但是这个也有对应的缺点,就是在线程池使用的时候不好使。所以通常最好的方使是使用阿里开源的transmittablethreadlocal来进行traceId的传递,切记使用完一定要记得将traceId clear掉。

日志

配置方式:
将自定义的链路 『trace_id』配置到 logback中(这里只配置控制台的方式,file文件配置方式一样)

  
        
            %d{yyyy-MM-dd HH:mm:ss.SSS} [%X{trace_id}] -[%thread] %-5level %logger{50}-%msg%n
        
    

我们模拟一个简单的请求代码:

@Slf4j
@RestController
public class TestController {

  @GetMapping("/test-single")
    public String testSingle() {
        String traceId = UUID.randomUUID().toString();
        MDC.put("trace_id", traceId);
        log.info("----main test--");
        handleBiz();
        return "error";
    }

    private void handleBiz() {
        log.info("----sub test--");
    }
}

然后得到正常的结果

2022-09-24 12:18:39.135 [26a63d09-9501-4d03-a61f-e09abf79dc7b] -[http-nio-1111-exec-1] INFO  com.lyqf.controller.TestController-----main test--
2022-09-24 12:18:39.135 [26a63d09-9501-4d03-a61f-e09abf79dc7b] -[http-nio-1111-exec-1] INFO  com.lyqf.controller.TestController-----sub test--

然而事事并非总是这么的如愿,如果我们的业务使用了异步线程,就不好使了,我们把代码改一下:

@Slf4j
@RestController
public class TestController {

  @GetMapping("/test-single")
    public String testSingle() {
        String traceId = UUID.randomUUID().toString();
        MDC.put("trace_id", traceId);
        log.info("----main test--");
        new Thread(() -> {
            handleBiz();
        }).start();
        return "error";
    }

    private void handleBiz() {
        log.info("----sub test--");
    }
}
2022-09-24 12:20:28.831 [07d0f883-0808-4b71-8c7d-356936a05ed8] -[http-nio-1111-exec-1] INFO  com.lyqf.controller.TestController-----main test--
2022-09-24 12:20:28.831 [] -[Thread-208] INFO  com.lyqf.controller.TestController-----sub test--

导致出现这个原因是logback里trace_id是获通过MDC来获取的,而MDC的实现方式是使用的ThreadLocal来实现的,所以在异步线程处理的时候获取到的traceId是空值。

总结一下:java异步通常用三种方式
1.直接创建单线程场景
2.使用CompletFuture及自定义线程场景
3.使用sping 及@Async 场景

后面我们将为每一种方式提供单独的解决方案。

解决方案

1.直接创建单线程场景

我们可以将traceId放在inheritableThreadLocal中,然后在异步线程中获取该值并放入MDC中从而实现。

@Slf4j
@RestController
public class TestController {
 private InheritableThreadLocal inheritableThreadLocal = new InheritableThreadLocal<>();
    
    @GetMapping("/test-single")
    public String testSingle() {
        String traceId = UUID.randomUUID().toString();
        MDC.put("trace_id", traceId);
        inheritableThreadLocal.set(traceId);
        log.info("----main test--");
        new Thread(() -> {
            try {
                MDC.put("trace_id", inheritableThreadLocal.get());
                log.info("----sub test--");
            } finally {
                MDC.clear();
            }
        }).start();

        inheritableThreadLocal.remove();
        return "error";
    }
}

执行结果

2022-09-23 17:27:00.038 [a9579bd9-c205-486a-a308-ab1c308d409b] -[http-nio-1111-exec-1] INFO  com.lyqf.controller.TestController-----main test--
2022-09-23 17:27:00.038 [a9579bd9-c205-486a-a308-ab1c308d409b] -[Thread-205] INFO  com.lyqf.controller.TestController-----sub test--
2. 使用CompletFuture及自定义线程场景

封装runnable、callable、Consumer任务类,为线程池提供使用。

public class MDCWrappers {

    public static Runnable wrap(final Runnable runnable) {
        final Map context = MDC.getCopyOfContextMap();
        return () -> {
            Map previous = MDC.getCopyOfContextMap();
            if (context == null) {
                MDC.clear();
            } else {
                MDC.setContextMap(context);
            }
            try {
                runnable.run();
            } finally {
                if (previous == null) {
                    MDC.clear();
                } else {
                    MDC.setContextMap(previous);
                }
            }
        };
    }

    public static  Callable wrap(final Callable callable) {
        final Map context = MDC.getCopyOfContextMap();
        return () -> {
            Map previous = MDC.getCopyOfContextMap();
            if (context == null) {
                MDC.clear();
            } else {
                MDC.setContextMap(context);
            }
            try {
                return callable.call();
            } finally {
                if (previous == null) {
                    MDC.clear();
                } else {
                    MDC.setContextMap(previous);
                }
            }
        };
    }

    public static  Consumer wrap(final Consumer consumer) {
        final Map context = MDC.getCopyOfContextMap();
        return (t) -> {
            Map previous = MDC.getCopyOfContextMap();
            if (context == null) {
                MDC.clear();
            } else {
                MDC.setContextMap(context);
            }
            try {
                consumer.accept(t);
            } finally {
                if (previous == null) {
                    MDC.clear();
                } else {
                    MDC.setContextMap(previous);
                }
            }
        };
    }

    public static  Collection> wrapCollection(Collection> tasks) {
        Collection> wrapped = new ArrayList<>();
        for (Callable task : tasks) {
            wrapped.add(wrap(task));
        }
        return wrapped;
    }
}

自定义MDCThreadPoolExecutor且实现ExecutorService接口

public class MDCThreadPoolExecutor implements ExecutorService {

    protected ExecutorService executorService;

    MDCThreadPoolExecutor() {
    }

    public MDCThreadPoolExecutor(
            int corePoolSize,
            int maximumPoolSize,
            long keepAliveTime,
            TimeUnit unit,
            BlockingQueue workQueue) {
        this.executorService = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    public MDCThreadPoolExecutor(
            int corePoolSize,
            int maximumPoolSize,
            long keepAliveTime,
            TimeUnit unit,
            BlockingQueue workQueue,
            ThreadFactory threadFactory) {
        this.executorService = new ThreadPoolExecutor(
                corePoolSize,
                maximumPoolSize,
                keepAliveTime,
                unit,
                workQueue,
                threadFactory);
    }

    public MDCThreadPoolExecutor(
            int corePoolSize,
            int maximumPoolSize,
            long keepAliveTime,
            TimeUnit unit,
            BlockingQueue workQueue,
            RejectedExecutionHandler handler) {
        this.executorService = new ThreadPoolExecutor(
                corePoolSize,
                maximumPoolSize,
                keepAliveTime,
                unit,
                workQueue,
                handler);
    }

    @Override
    public  Future submit(Callable task) {
        return executorService.submit(MDCWrappers.wrap(task));
    }

    @Override
    public  Future submit(Runnable task, T result) {
        return executorService.submit(MDCWrappers.wrap(task), result);
    }

    @Override
    public Future submit(Runnable task) {
        return executorService.submit(MDCWrappers.wrap(task));
    }

    @Override
    public  List> invokeAll(Collection> tasks) throws InterruptedException {
        return executorService.invokeAll(MDCWrappers.wrapCollection(tasks));
    }

    @Override
    public  List> invokeAll(Collection> tasks, long timeout, TimeUnit unit)
            throws InterruptedException {
        return executorService.invokeAll(MDCWrappers.wrapCollection(tasks), timeout, unit);
    }

    @Override
    public  T invokeAny(Collection> tasks) throws InterruptedException, ExecutionException {
        return executorService.invokeAny(MDCWrappers.wrapCollection(tasks));
    }

    @Override
    public  T invokeAny(Collection> tasks, long timeout, TimeUnit unit)
            throws InterruptedException, ExecutionException, TimeoutException {
        return executorService.invokeAny(MDCWrappers.wrapCollection(tasks), timeout, unit);
    }

    @Override
    public void execute(Runnable command) {
        executorService.execute(MDCWrappers.wrap(command));
    }

    @Override
    public void shutdown() {
        executorService.shutdown();
    }

    @Override
    public List shutdownNow() {
        return executorService.shutdownNow();
    }

    @Override
    public boolean isShutdown() {
        return executorService.isShutdown();
    }

    @Override
    public boolean isTerminated() {
        return executorService.isTerminated();
    }

    @Override
    public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {
        return executorService.awaitTermination(timeout, unit);
    }
}

请求代码块

@Slf4j
@RestController
public class TestController {
 MDCThreadPoolExecutor threadPoolExecutor =
        new MDCThreadPoolExecutor(1, 1, 1, TimeUnit.HOURS, new ArrayBlockingQueue<>(10));

    MDCThreadPoolExecutor completableFutureThreadPool =
        new MDCThreadPoolExecutor(1, 1, 1, TimeUnit.HOURS, new ArrayBlockingQueue<>(10));

    @GetMapping("/test-thread-pool")
    public String testThreadPool() {
        MDC.put("trace_id", UUID.randomUUID().toString());
        log.info("----main test--");
        threadPoolExecutor.execute(() -> {
            log.info("----sub threadPoolExecutor  test--");
        });
        CompletableFuture.supplyAsync(() -> {
            log.info("----sub CompletableFuture  test--");
            return "success";
        }, completableFutureThreadPool);
        return "success";
    }
}

输出结果:

2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[http-nio-1111-exec-2] INFO  com.lyqf.controller.TestController-----main test--
2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[pool-5-thread-1] INFO  com.lyqf.controller.TestController-----sub CompletableFuture  test--
2022-09-24 11:00:48.218 [36d492c0-8937-4263-b842-4268d0de728d] -[pool-4-thread-1] INFO  com.lyqf.controller.TestController-----sub threadPoolExecutor  test--


3.使用sping 及@Async 场景
public class MdcTaskDecorator implements TaskDecorator {

    @Override
    public Runnable decorate(Runnable runnable) {
        Map contextMap = MDC.getCopyOfContextMap();
        return () -> {
            try {
                MDC.setContextMap(contextMap);
                runnable.run();
            } finally {
                MDC.clear();
            }
        };
    }
}

配置Bean

@Configuration
public class MdcConfig {
    @Bean
    public AsyncTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        // 设置线程名称
        executor.setThreadNamePrefix("customer-executor");
        // 设置最大线程数
        executor.setMaxPoolSize(50);
        // 设置核心线程数
        executor.setCorePoolSize(10);
        // 设置线程空闲时间,默认60
        executor.setKeepAliveSeconds(60);
        // 设置队列容量
        executor.setQueueCapacity(1000);
        executor.setTaskDecorator(new MdcTaskDecorator());
        // 设置拒绝策略
        executor.setRejectedExecutionHandler(new RejectedExecutionHandler() {
            @Override
            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
                //todo
            }
        });
        return executor;
    }
}

构建请求controller

@Slf4j
@RestController
public class TestController {
 @Resource
    private TestService testService;

    @GetMapping("/test")
    public String test() {
        MDC.put("trace_id", UUID.randomUUID().toString());
        log.info("----main test--");
        testService.test();
        return "success";
    }

异步代码块

@Slf4j
@Service
public class TestService {
    @Async
    public  void test() {
        log.info("----TestService test--");
    }
}

输出结果

2022-09-23 17:11:12.190 [311dd086d94246e4b819c79bec72350f] -[http-nio-1111-exec-2] INFO  com.lyqf.controller.TestController-----main test--
2022-09-23 17:11:12.191 [311dd086d94246e4b819c79bec72350f] -[customer-executor2] INFO  com.lyqf.TestService-----TestService test--

你可能感兴趣的:(聊一聊链路日志-异步线程)