logback实现日志脱敏

背景

因客户特殊性，在线数据日志需要做脱敏处理，业务代码层实现需要调整所有logger.info，代码侵入性大且修改成本高，本次实现通过自定义日志layout，定义对应正则匹配表达式，对打印message进行替换处理，同时支持Nacos配置动态刷新

实现

1. 实现自定义Layout

   • doLayout()实现对日志的正则匹配和替换

   • 定义静态变量newFilterPropertyNames（因每次打印初始化对象，使用静态变量来共用配置变量），支持配置类动态刷新配置后同步修改

   • 正则表达式"${fieldname}":"[\u4e00-\u9fa5\w\s-·•\/]+"，支持匹配json字符传中的字段属性包含汉字、字母、数字、下划线、空白字符、-·•/，后续涉及字符再同步修改

/**
* @author zhaoccf
* @version 1.0
* @description 自定义日志信息处理，进行名称脱敏处理
*/
public class SensitiveFilterPatternLayout extends PatternLayout {
   private static final String REGEX_PRE = """;
   private static final String REGEX_SUF = "":"[\\u4e00-\\u9fa5\\w\\s-·•\\/]+"";
   private static final String REGEX_COMMA = ",{2,}";
   private static final String REGEX_COMMA_PRE = "\\{,";
   private static final String REGEX_COMMA_SUF = ",\\}";
   private static List patterns = new ArrayList<>();
   private static String filterPropertyNames;
   public static String newFilterPropertyNames;

   public SensitiveFilterPatternLayout() {
       //patterns已初始化完成并且未修改配置文件，直接返回
       if (!ObjectUtils.isEmpty(newFilterPropertyNames) && newFilterPropertyNames.equals(filterPropertyNames)) {
           return;
       }
       //配置文件未加载
       if (ObjectUtils.isEmpty(newFilterPropertyNames) && ObjectUtils.isEmpty(filterPropertyNames)) {
           return;
       }
       //开始初始化patterns，配置修改newFilterPropertyNames后重新设置filterPropertyNames用于初始化判断并更新正则表达式patterns
       if (!ObjectUtils.isEmpty(newFilterPropertyNames)) {
           filterPropertyNames = newFilterPropertyNames;
           patterns.clear();
       }
       System.out.println("init filterPropertyNames:".concat(filterPropertyNames));

       //拼接匹配正则表达式
       Arrays.asList(filterPropertyNames.split(",")).stream().forEach(name -> {
           String regexStr = new StringBuffer(REGEX_PRE).append(name).append(REGEX_SUF).toString();
           patterns.add(Pattern.compile(regexStr));
       });
   }

   @Override
   public String doLayout(ILoggingEvent event) {
       String message = super.doLayout(event);
       Set matches = new HashSet<>();
       for (Pattern pattern : patterns) {
           Matcher matcher = pattern.matcher(message);
           while (matcher.find()) {
               for (int i = 0; i <= matcher.groupCount(); i++) {
                   matches.add(matcher.group(i));
               }
           }
       }
       //因为字段可能在最后一个也可能不在，存在{"productSortName":"分配zcc"}和{"productSortName":"分配zcc",}两种匹配项导致替换后出现多个逗号情况
       //如果为""不做过滤替换因为没有数据
       for (String match : matches) {
           message = message.replaceAll(match, "");
       }
       Pattern comma = Pattern.compile(REGEX_COMMA);
       Matcher commaMatcher = comma.matcher(message);
       if (commaMatcher.find()) {
           message =  commaMatcher.replaceAll(",");
       }

       Pattern commaPre = Pattern.compile(REGEX_COMMA_PRE);
       Matcher commaPreMatcher = commaPre.matcher(message);
       if (commaPreMatcher.find()) {
           message =  commaPreMatcher.replaceAll("{");
       }

       Pattern commaSuf = Pattern.compile(REGEX_COMMA_SUF);
       Matcher commaSufMatcher = commaSuf.matcher(message);
       if (commaSufMatcher.find()) {
           message =  commaSufMatcher.replaceAll("}");
       }
       return message;
   }

   private boolean containFilterName(String msg) {
       return Arrays.asList(filterPropertyNames.split(",")).stream().anyMatch(item -> msg.contains(item));
   }
}

2. 修改logback配置文件（默认文件logback-spring.xml, logback-spring.groovy, logback.xml, or logback.groovy）的encoder，并添加自定义Layout为其变量

   
   
       
           ${CONSOLE_LOG_PATTERN}
       
       
       utf-8
   
   
   
       debug
   

3. 添加配置类并支持Nacos动态配置刷新（@RefreshScope），并在配置类初始化时更新自定义Layout的字段名配置变量

/**
* @author zhaoccf
* @version 1.0
* @description 日志脱敏处理过滤字段支持nacos动态配置配置类
*/
@Configuration
@RefreshScope
@Slf4j
public class SensitiveFilterLogConfiguration {

   @Value("${log.filter.propertyNames}")
   private String filterPropertyNames;

   @PostConstruct
   private void init() {
       if (!StringUtils.isEmpty(filterPropertyNames)) {
           if (log.isInfoEnabled()) {
               log.info("SensitiveFilterLogConfiguration refresh filterPropertyNames:{}", filterPropertyNames);
           }
           SensitiveFilterPatternLayout.newFilterPropertyNames = filterPropertyNames;
       }
   }
}

4. Nacos配置中心添加配置项

log:
 filter:
   propertyNames: fundName,fundShortName,masterFundName,managerName,productName,productShortName,companyName,dealName,leadOperator

待优化项：

1. 配置加载方式可以更优雅
2. 正则匹配字符不完善
3. 脱敏方式可修改替换为打*
4. 每次打印会重新初始化Layout，可以研究下lagback源码，寻找lagback周期中脱敏处理的最佳时机

参考链接：Mask sensitive data in logs with logback - Stack Overflow