将 tomcat 日志记录成 json

本文 Tomcat 版本为 8.0.36。

首先要搞清楚一点，开发打的业务日志要么让开发打成 json 格式，要么使用 Logstash 进行解析，不可能有通用的产品可以完成，因为每个公司开发打印的日志的标准都不一样。这里改变的只是 tomcat 本身的日志格式。

tomcat 自身的日志有：

catalina.2018-01-05.log
host-manager.2018-01-05.log
localhost.2018-01-05.log
manager.2018-01-05.log
localhost_access_log.2018-01-05.log

catalina.out 包括 tomcat 本身和开发打印的应用日志，不做讨论。

Tomcat 日志的访问日志是在 server.xml 中配置的，配置成 json 格式也十分方便，而对于其他日志想要配置成 json 格式就十分不便了。对于这些日志，tomcat 提供了两种记录的方式，一种是默认的 java.util.logging，另一种就是 log4j。默认的方式无法记录成 json（至少我没找到方式），而通过 log4j 可以，因为 Logstash 提供了一个插件。

首先我们要做的就是将 tomcat 日志的记录的方式改为 log4j，修改的方式在此。这是官方文档的翻译，你也可以直接看官方文档。

需要注意的是：

• log4j 的 jar 包一定要是 1.2.X 版本的，不要下载最新版；
• 示例的 log4j.properties 中指定的日志文件是 catalina，而非 catalina.out。

所有 jar 包都可以在 https://mvnrepository.com/ 这里下载。

如果没有改变官方给出的 log4j.properties 的日志记录格式，那么配置成功后，tomcat/logs/catalina 的输出和之前的 catalina.out 一样。下一步就是使用 Logstash 提供的插件，这个怎么用呢？很简单，首先去上面给出的 maven 仓库中下载 jar 包（最好使用 1.6 版），然后放入 tomcat/lib 目录下，接着修改 log4j.properties：

log4j.rootCategory=WARN, RollingLog
log4j.appender.RollingLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RollingLog.Threshold=TRACE
log4j.appender.RollingLog.File=api.log
log4j.appender.RollingLog.DatePattern=.yyyy-MM-dd
log4j.appender.RollingLog.layout=net.logstash.log4j.JSONEventLayoutV1

为了可以直接效果，可以将日志级别改为 INFO。这个时候如果直接使用的话，会有下面的报错：

Using CATALINA_BASE:   /tmp/tomcat
Using CATALINA_HOME:   /tmp/tomcat
Using CATALINA_TMPDIR: /tmp/tomcat/temp
Using JRE_HOME:        /usr/local/jdk1.7.0_79
Using CLASSPATH:       /tmp/tomcat/bin/bootstrap.jar:/tmp/tomcat/bin/tomcat-juli.jar
log4j:WARN Error during default initialization
java.lang.NoClassDefFoundError: org/apache/commons/lang/time/FastDateFormat
    at net.logstash.log4j.JSONEventLayoutV0.(JSONEventLayoutV0.java:36)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at java.lang.Class.newInstance(Class.java:379)
    at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:336)
    at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:123)
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:764)
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
    at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
    at org.apache.log4j.LogManager.(LogManager.java:127)
    at org.apache.log4j.Logger.getLogger(Logger.java:104)
    at org.apache.juli.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262)
    at org.apache.juli.logging.impl.Log4JLogger.(Log4JLogger.java:108)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.juli.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025)
    at org.apache.juli.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:844)
    at org.apache.juli.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541)
    at org.apache.juli.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292)
    at org.apache.juli.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269)
    at org.apache.juli.logging.LogFactory.getLog(LogFactory.java:657)
    at org.apache.catalina.startup.Catalina.(Catalina.java:820)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at java.lang.Class.newInstance(Class.java:379)
    at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:268)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:455)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.time.FastDateFormat
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 36 more

需要下载以下包，放入 tomcat/lib 目录下：

commons-lang-2.4
json-smart-1.1.1
junit-4.8.1

然后就可以在 api.log 中看到效果了：

{"@fields":{"level":"INFO","threadName":"localhost-startStop-1","mdc":{},"file":"HostConfig.java","class":"org.apache.catalina.startup.HostConfig","line_number":"1143","method":"deployDirectory","loggerName":"org.apache.catalina.startup.HostConfig"},"@timestamp":"2018-01-05T01:36:38.263Z","@message":"Deployment of web application directory \/tmp\/tomcat\/webapps\/manager has finished in 64 ms","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"AbstractProtocol.java","class":"org.apache.coyote.AbstractProtocol","line_number":"471","method":"start","loggerName":"org.apache.coyote.http11.Http11NioProtocol"},"@timestamp":"2018-01-05T01:36:38.265Z","@message":"Starting ProtocolHandler [\"http-nio-9021\"]","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"AbstractProtocol.java","class":"org.apache.coyote.AbstractProtocol","line_number":"471","method":"start","loggerName":"org.apache.coyote.ajp.AjpNioProtocol"},"@timestamp":"2018-01-05T01:36:38.270Z","@message":"Starting ProtocolHandler [\"ajp-nio-8009\"]","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"Catalina.java","class":"org.apache.catalina.startup.Catalina","line_number":"642","method":"start","loggerName":"org.apache.catalina.startup.Catalina"},"@timestamp":"2018-01-05T01:36:38.272Z","@message":"Server startup in 695 ms","@source_host":"SUC-IAS-JOB-02.UAT"}