logback + nginx 实现最简单的日志框架
logback + nginx 实现最简单的日志框架
由来
经理： 咱们现在业务量增加了很多，问题定位需要输出大量的日志！
码农：可以部署 elastic stock 分布式日志框架，它可以储存PB级别的日志 ……
经理：咱们这东西很好但是有一定的学习成本，部署也相对麻烦！
码农：可以容器化部署，或者ansible，jenkins 一键式部署 ……
经理: 嗯，但是我就是想要个hello world 能比较方便的查询到系统日志文件就好！
码农：嗯，那好的！
于是乎， logback + nginx 日志框架诞生。（最终目标，logback + netty +web,实现日志收集和实时推送，报警，储存，分级，分类等）。
框架
首先准备Nginx
技术选型原因
很简单，说到对静态资源文件的解析，nginx无出其右，当然使用live-server之类的可以实现更多功能，例如：实时热重载，日志文件发生修改实时推送给页面。但是文件监听是有消耗的，而日志文件体量很大，因此这里首选nginx，还有个原因是，配置部署简单，本身对nginx也有些粗浅的了解。
首先下载nginx（本文以windows为例，因为测试简单）
下载地址：
http://nginx.org/en/download.html
安装，不用安装，不解释。
配置nginx为文件服务器
主要配置：
allow all;
autoindex on;
autoindex_exact_size off;
autoindex_localtime on;
- autoindex on 表示显示目录
- autoindex_exact_size 显示文件大小显示单位不一样而已，可以自己实验一下
- autoindex_localtime 显示文件时间
设置日志文件目录为nginx资源目录
root D:/log/project/;
还有一点就是编码格式，否则会出现日志乱码
charset utf-8;
我的配置文件如下：
然后启动就可以了。
这个时候，nginx，已经完成了，但是如果你想要是通过浏览器查询日志，而不是下载日志（我的需求是查看，而不是下载），并且你输出的日志拓展名为 .log ,那么你需要加上如下配置，将.log文件声明为文本文件。
最简单的方式，添加一条mimeType的声明：
conf/mime.types 文件中添加
text/plain log;
配置截图如下：
配置logback
logback 添加文件输出的Appender
配置如下：
<appender name="errorFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>trueprudent>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LogFilePath}/error-log/error-%d{yyyy-MM-dd}.txtfileNamePattern>
<maxHistory>72maxHistory>
<totalSizeCap>3GBtotalSizeCap>
rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>{"tags":"controller", "timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss,SSSZ}", "level": "%level", "thread": "%thread", "location": "%logger{0}::%method", "message": "%message"}%npattern>
encoder>
appender>
简单的说明一下：
- TimeBasedRollingPolicy 配置什么时候触发，并且按照时间声明文件名%d{yyyy-MM-dd} 为声明到天，%d{yyyy-MM-dd HH}声明到小时，以此类推。
- pattern 控制处处内容，这里我希望输出json，但是又懒得去自定义一个layout 所以直接使用了字符串的方式。
- ${LogFilePath} 是定义的基础日志输出位置变量 声明方式为 ：
<property name="LogFilePath" value="D:/log/project/schedule"/>
注：这里的路径一定为nginx资源目录，或者是子目录，否则需要添加新的nginx配置才能监听到这些日志文件。
####### 定义logger 输出到自定文件
这里以错误日志输出为例（上面也是错误日志的appender声明）
<logger name="com.guodun" level="error">
<appender-ref ref="errorFILE"/>
logger>
到这里logback 已经准备完成了。下面的为一些建议和自己的拙见
如果，你也像我一样有点代码洁癖，如果也需要很多的不同类型的日志文件输出，同时又不想让logback的配置文件过于臃肿，那么就添加一个appender的声明配置文件吧。
引用语法：
<include resource="logback-file-appenders.xml"/>
声明语法：
<included>
<appender name="includedConsole" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d - %m%npattern>
encoder>
appender>
included>
被引用的配置文件，必须以 included 标签包裹
注：以上引用为引用资源文件下的配置，也是比较常用的，还有些其他的引用，如下
引用指定文件
<include file="src/main/resources/includedConfig.xml" />
引用网络配置:
<include url="http://some.host.com/includedConfig.xml"/>
运行结果如下：
打开日志文件截图：
附上源码：
logback.xml
<configuration>
<include resource="logback-file-appenders.xml"/>
<conversionRule conversionWord="customcolor" converterClass="com.guodun.serve.schedule.config.LogbackColorConfig"/>
<property resource="application.properties" />
<property name="jsonLog" value="{time:%d{yyyy-MM-dd HH:mm:ss.SSS},location: %logger{0}::%method, msg:%msg}%n"/>
<appender name="LOGSTASH"
class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>192.168.1.141:9601destination>
<encoder charset="UTF-8"
class="net.logstash.logback.encoder.LogstashEncoder" >
<customFields>{"appname":"${spring.application.name}","IP":"localhost","port":"${server.port}"}customFields>
encoder>
appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS}%customcolor([%thread]) %customcolor(%-5level) %logger{50} - %customcolor(%msg) %npattern>
encoder>
appender>
<appender name="ControllerConsole" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>[ %d{HH:mm:ss.SSS} ] %customcolor(%-5level)| %logger{0} :: %method | - %customcolor(%msg) %npattern>
encoder>
appender>
<root level="info">
<appender-ref ref="STDOUT"/>
root>
<logger name="com.guodun.serve.schedule" level="debug">
<appender-ref ref="ControllerConsole"/>
logger>
<logger name="com.guodun" level="info">
<appender-ref ref="LOGSTASH"/>
logger>
<logger name="com.guodun" level="error">
<appender-ref ref="errorFILE"/>
logger>
<logger name="com.guodun.serve.schedule.strategy" level="debug">
<appender-ref ref="strategyFILE"/>
logger>
<logger name="com.guodun.serve.schedule.controller" level="debug">
<appender-ref ref="ControlleFILE"/>
logger>
<logger name="com.netflix" level="OFF">logger>
<logger name="o.a.c.core" level="OFF">logger>
<logger name="org.springframework.web" level="OFF">logger>
configuration>
logback-file-appenders.xml
<included>
<property name="LogFilePath" value="D:/log/guodun/schedule"/>
<appender name="errorFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>trueprudent>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LogFilePath}/error-log/error-%d{yyyy-MM-dd}.txtfileNamePattern>
<maxHistory>72maxHistory>
<totalSizeCap>3GBtotalSizeCap>
rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>{"tags":"controller", "timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss,SSSZ}", "level": "%level", "thread": "%thread", "location": "%logger{0}::%method", "message": "%message"}%npattern>
encoder>
appender>
<appender name="strategyFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>trueprudent>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LogFilePath}/strategy-log/log-%d{yyyy-MM-dd HH}.txtfileNamePattern>
<maxHistory>72maxHistory>
<totalSizeCap>3GBtotalSizeCap>
rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>{"tags":"controller", "timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss,SSSZ}", "level": "%level", "thread": "%thread", "location": "%logger{0}::%method", "message": "%message"}%npattern>
encoder>
appender>
<appender name="ControlleFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>trueprudent>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LogFilePath}/controller-log/ControllerLog-%d{yyyy-MM-dd HH}.txtfileNamePattern>
<maxHistory>72maxHistory>
<totalSizeCap>3GBtotalSizeCap>
rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>{"tags":"controller", "timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss,SSSZ}", "level": "%level", "thread": "%thread", "location": "%logger{0}::%method", "message": "%message"}%npattern>
encoder>
appender>
included>
mime.types
types {
text/html html htm shtml;
text/css css;
text/xml xml;
image/gif gif;
image/jpeg jpeg jpg;
application/javascript js;
application/atom+xml atom;
application/rss+xml rss;
text/mathml mml;
text/plain txt;
text/plain log;
text/vnd.sun.j2me.app-descriptor jad;
text/vnd.wap.wml wml;
text/x-component htc;
image/png png;
image/svg+xml svg svgz;
image/tiff tif tiff;
image/vnd.wap.wbmp wbmp;
image/webp webp;
image/x-icon ico;
image/x-jng jng;
image/x-ms-bmp bmp;
font/woff woff;
font/woff2 woff2;
application/java-archive jar war ear;
application/json json;
application/mac-binhex40 hqx;
application/msword doc;
application/pdf pdf;
application/postscript ps eps ai;
application/rtf rtf;
application/vnd.apple.mpegurl m3u8;
application/vnd.google-earth.kml+xml kml;
application/vnd.google-earth.kmz kmz;
application/vnd.ms-excel xls;
application/vnd.ms-fontobject eot;
application/vnd.ms-powerpoint ppt;
application/vnd.oasis.opendocument.graphics odg;
application/vnd.oasis.opendocument.presentation odp;
application/vnd.oasis.opendocument.spreadsheet ods;
application/vnd.oasis.opendocument.text odt;
application/vnd.openxmlformats-officedocument.presentationml.presentation
pptx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
xlsx;
application/vnd.openxmlformats-officedocument.wordprocessingml.document
docx;
application/vnd.wap.wmlc wmlc;
application/x-7z-compressed 7z;
application/x-cocoa cco;
application/x-java-archive-diff jardiff;
application/x-java-jnlp-file jnlp;
application/x-makeself run;
application/x-perl pl pm;
application/x-pilot prc pdb;
application/x-rar-compressed rar;
application/x-redhat-package-manager rpm;
application/x-sea sea;
application/x-shockwave-flash swf;
application/x-stuffit sit;
application/x-tcl tcl tk;
application/x-x509-ca-cert der pem crt;
application/x-xpinstall xpi;
application/xhtml+xml xhtml;
application/xspf+xml xspf;
application/zip zip;
application/octet-stream bin exe dll;
application/octet-stream deb;
application/octet-stream dmg;
application/octet-stream iso img;
application/octet-stream msi msp msm;
audio/midi mid midi kar;
audio/mpeg mp3;
audio/ogg ogg;
audio/x-m4a m4a;
audio/x-realaudio ra;
video/3gpp 3gpp 3gp;
video/mp2t ts;
video/mp4 mp4;
video/mpeg mpeg mpg;
video/quicktime mov;
video/webm webm;
video/x-flv flv;
video/x-m4v m4v;
video/x-mng mng;
video/x-ms-asf asx asf;
video/x-ms-wmv wmv;
video/x-msvideo avi;
}
nginx.conf
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
server {
listen 6780;
server_name localhost;
charset utf-8;
#access_log logs/host.access.log main;
location / {
root D:/log/guodun/;
allow all;
autoindex on;
autoindex_exact_size off;
autoindex_localtime on;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443 ssl;
# server_name localhost;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 5m;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
时间比较紧张，有些地方不完善，望请见谅。
如果感觉有用，点个赞再走呗。
----------------------------------------------------------这就是我的底线--------------------------------------------------------------