mysql5.7 中启用MTS后error log中大量Note日志分析

mysql5.7,启用基于logical_clock的多线程复制,发现error日志增长很快，查看日志发现大量关于多线程复制的Note级别日志。

2018-07-03T03:22:01.638371+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 298; events assigned = 3043329; worker queues filled over ove
rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725810947700 waited (count) when Workers occupied = 5346 wait
ed when Workers occupied = 0
2018-07-03T03:24:22.589147+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 141; events assigned = 3044353; worker queues filled over ove
rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725810947700 waited (count) when Workers occupied = 5346 wait
ed when Workers occupied = 0
2018-07-03T03:27:00.554437+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 158; events assigned = 3045377; worker queues filled over ove
rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725818557700 waited (count) when Workers occupied = 5346 wait
ed when Workers occupied = 0
2018-07-03T03:32:00.079699+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 300; events assigned = 3047425; worker queues filled over ove
rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725846344900 waited (count) when Workers occupied = 5346 wait
ed when Workers occupied = 0
2018-07-03T03:34:04.567887+08:00 8941 [Note] Multi-threaded slave statistics for channel '': seconds elapsed = 124; events assigned = 3048449; worker queues filled over ove
rrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 725852036800 waited (count) when Workers occupied = 5346 wait
ed when Workers occupied = 0

通过日期可以看到，日志的打印频率大概在3分钟左右。error log是mysql 出现问题时的重要分析日志，大量的note日志可能很快就把重要日志信息埋没,给分析日志信息带来了不便。而且日志量随时间递增，还必须有rotate机制，不然占用大量磁盘空间。那么这个日志的打印频率能不能调低一些呢？

我看了下源码这块的实现(rpl_slave.cc 4800)

  *ptr_ev= NULL; // announcing the event is passed to w-worker
 
       if (rli->is_parallel_exec() && rli->mts_events_assigned % 1024 == 1)
       {
         time_t my_now= my_time(0);
 
         if ((my_now - rli->mts_last_online_stat) >=
             mts_online_stat_period)
         {
           sql_print_information("Multi-threaded slave statistics%s: "
                                 "seconds elapsed = %lu; "
                                 "events assigned = %llu; "
                                 "worker queues filled over overrun level = %lu; "
                                 "waited due a Worker queue full = %lu; "
                                 "waited due the total size = %lu; "
                                 "waited at clock conflicts = %llu "
                                 "waited (count) when Workers occupied = %lu "
                                 "waited when Workers occupied = %llu",
                                 rli->get_for_channel_str(),
                                 static_cast
                                 (my_now - rli->mts_last_online_stat),
                                 rli->mts_events_assigned,
                                 rli->mts_wq_overrun_cnt,
                                 rli->mts_wq_overfill_cnt,
                                 rli->wq_size_waits_cnt,
                                 rli->mts_total_wait_overlap,
                                 rli->mts_wq_no_underrun_cnt,
                                 rli->mts_total_wait_worker_avail);
           rli->mts_last_online_stat= my_now;

我们从两个if语句可以看到，这个信息的打印受三个条件的控制：

1. rli->is_parallel_exec() 这是个inline函数，它判断是否在使用多线程复制
2. rli->mts_events_assigned % 1024 == 1 多线程执行的event个数刚刚超过1024个。也就是说按evnet个数来算，每隔1024个event打印一次。
3. 最后还有个时间限制：(my_now - rli->mts_last_online_stat) >=
   mts_online_stat_period，距离上一次统计超过既定的间隔mts_online_stat_period。这个变量是60*2 ,代码里写死了。

 /*
   Statistics go to the error log every # of seconds when --log-warnings > 1
 */
 const long mts_online_stat_period= 60 * 2;

由此可以判断，这个日志打印频率最少为2分钟，没法改了。但上面的代码注释中给了提示，当--log-warnings > 1时，这个统计间隔才生效，意味着可以通过修改日志打印级别来控制。

自mysql5.7.2，log-warnings被废弃，引用了新的变量log_error_verbosity。这个变量有三个值：1、2、3，默认是3. 他们的意义是：
1 -- Errors Only
2 -- Errors and warnings
3 -- Errors, warnings, and notes

我们可以修改这个变量为2 ，不再打印Note信息。

set global log_error_verbosity=2;

阅读更多