【10.17】nginx 访问日志、静态文件不记录、过期缓存、日志切割、防盗链
【10.17】nginx 访问日志、静态文件不记录、过期缓存、日志切割、防盗链
- 4.38 nginx访问日志
- 4.39 日志不记录静态文件
- 4.40 日志切割
- 4.41 静态文件过期缓存
- 4.42 nginx防盗链
4.38 nginx访问日志
访问日志：用户访问网站的记录
主要就是配置路径加格式
- 1、配置方法：
1)、主配置文件 /etc/nginx/nginx.conf 中，定义日志格式
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
对应变量含义：
| 变量 | 含义 |
|---|---|
| $remote_addr | 客户端IP(公网IP) |
| $http_x_forwarded_for | 代理服务器的IP |
| $time_local | 服务器本地时间 |
| $host | 访问主机名(域名) |
| $request_uri | 访问的url地址 |
| $status | 状态码 |
| $http_referer | referer |
| $http_user_agent | user_agent |
长用全局变量汇总：https://blog.csdn.net/arsenal4life/article/details/102616484
2)、虚拟主机配置 /etc/nginx/conf.d/bbs.ars4life.com.conf 的最下方，加入一行 access_log
access_log /data/logs/bbs.ars4life.com.access.log main;
用于定义访问日志
2、改好配置后，需要创建 /data/logs/ 目录，否则会报错
[root@alexis-01 ~]# mkdir /data/logs/
[root@alexis-01 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@alexis-01 ~]# nginx -s reload
[root@alexis-01 ~]# cat /data/logs/bbs.ars4life.com.access.log
3、访问网页后，日志会产生以下内容
[root@alexis-01 ~]# cat /data/logs/bbs.ars4life.com.access.log
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 200 26614 "http://444444.com/forum.php?gid=1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /data/cache/style_1_forum_forumdisplay.css?fv2 HTTP/1.1" 304 0 "http://444444.com/forum.php?mod=forumdisplay&fid=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:22:13:47 +0800] "GET /static/image/common/fall.png HTTP/1.1" 301 169 "http://444444.com/data/cache/style_1_forum_forumdisplay.css?fv2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /static/image/common/fall.png HTTP/1.1" 200 843 "http://444444.com/data/cache/style_1_forum_forumdisplay.css?fv2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /misc.php?mod=seccode&action=update&idhash=cSgMt1Yd&0.14961814848941724&modid=forum::forumdisplay HTTP/1.1" 200 1553 "http://444444.com/forum.php?mod=forumdisplay&fid=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /misc.php?mod=seccode&update=44442&idhash=cSgMt1Yd HTTP/1.1" 200 665 "http://444444.com/forum.php?mod=forumdisplay&fid=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - user1 [17/Oct/2019:22:13:47 +0800] "GET /misc.php?mod=patch&action=pluginnotice&inajax=1&ajaxtarget=plugin_notice HTTP/1.1" 200 76 "http://444444.com/forum.php?mod=forumdisplay&fid=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
4.39 日志不记录静态文件
在访问日志里，过滤掉一些图片、js、css类的请求日志
因为此类请求没有多大用处，而且会占用很大的磁盘空间
- 配置方法：
/etc/nginx/conf.d/bbs.ars4life.com.conf 里 server_name 下增加一段 location，如下
location ~* \.(png|jpeg|gif|js|css|bmp|flv)
{
access_log off;
}
清空访问日志，再去访问网页，之后查看访问日志，就不会出现以上后缀名的记录了
[root@alexis-01 ~]# tail /data/logs/bbs.ars4life.com.access.log
192.168.194.1 - - [17/Oct/2019:23:04:24 +0800] "GET /install/index.php HTTP/1.1" 200 1345 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:23:04:25 +0800] "GET /favicon.ico HTTP/1.1" 304 0 "http://bbs.ars4life.com/install/index.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:23:04:59 +0800] "GET /favicon.ico HTTP/1.1" 200 5558 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:23:05:00 +0800] "GET / HTTP/1.1" 200 12310 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:23:05:00 +0800] "GET /home.php?mod=misc&ac=sendmail&rand=1571324700 HTTP/1.1" 200 5 "http://bbs.ars4life.com/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"
192.168.194.1 - - [17/Oct/2019:23:05:00 +0800] "GET / HTTP/1.1" 200 12137 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"
补充：
tail -f /data/logs/bbs.ars4life.com // -f 可以动态查看一个文件的内容
> 可以清空一个文件内容
~* 表示不区分大小写的匹配，后面跟正则表达式
. 表示任意字符
4.40 日志切割
日志切割目的： 访问日志有很多，日志会越来越大，可能有一天就会把整个磁盘写满，100G的日志日和查看？？？
- 系统里有个日志切割的工具： logrotate
- logrotate 配置文件： /etc/logrotate.conf
- 子配置文件： /etc/logrotate.d/*
[root@alexis-01 ~]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
dateext
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0600 root utmp
rotate 1
}
# system-specific logs may be also be configured here.
- Nginx 日志切割配置文件：
/etc/logrotate.d/nginx - 如果是 yum 安装的 nginx，那么再 /etc/logrotate.d/nginx 中已经有 nginx 切割策略
[root@alexis-01 ~]# cat /etc/logrotate.d/nginx
/var/log/nginx/*.log /data/logs/*.log{
daily
dateext
missingok
rotate 7
compress
delaycompress
notifempty
create 640 nginx adm
sharedscripts
postrotate
if [ -f /var/run/nginx.pid ]; then
kill -USR1 `cat /var/run/nginx.pid`
fi
endscript
}
daily 每天
dateext 以日期为后缀
missingok 忽略错误
rotate 7 保留7天
compress 压缩
delaycompress 延后压缩（今天切割，昨天有一个文件，前天的文件可以压缩，昨天的不压缩）
notifempty 如果是空文件，不进行切割
create 640 nginx adm 创建回滚文件的权限，用户和用户组
sharedscripts 执行脚本
- 测试执行 logrotate -v /etc/logrotate.d/nginx
- 强制切割 logrotate -vf /etc/logrotate.d/nginx
[root@alexis-01 ~]# logrotate -v /etc/logrotate.d/nginx
reading config file /etc/logrotate.d/nginx
Allocating hash table for state file, size 15360 B
Handling 1 logs
rotating pattern: /var/log/nginx/*.log after 1 days (52 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/nginx/access.log
log does not need rotating (log has been rotated at 2019-10-17 12:12, that is not day ago yet)
considering log /var/log/nginx/error.log
log does not need rotating (log has been rotated at 2019-10-17 12:12, that is not day ago yet)
not running postrotate script, since no logs were rotated
set default create context
[root@alexis-01 ~]# ll /data/logs/
总用量 4
-rw-r--r--. 1 root root 1648 10月 17 23:11 bbs.ars4life.com.access.log
[root@alexis-01 ~]# ll /var/log/nginx/
总用量 284
-rw-r-----. 1 nginx adm 98028 10月 17 23:05 access.log
-rw-r-----. 1 nginx adm 88509 10月 17 12:06 access.log-20191017
-rw-r-----. 1 nginx adm 22806 10月 17 23:05 error.log
-rw-r-----. 1 nginx adm 77693 10月 17 12:05 error.log-20191017
/var/log/nginx 下的日志已切割
4.41 静态文件过期缓存
服务端静态文件已经更新了，但是一直在请求旧的缓存，这样不合理，需要设定过期时间
- 先查看bbs.ars4life.com上图片的信息
[root@alexis-01 ~]# curl -x127.0.0.1:80 -I http://bbs.ars4life.com/static/image/common/logo_88_31.gif
HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Fri, 18 Oct 2019 13:05:36 GMT
Content-Type: image/gif
Content-Length: 2528
Last-Modified: Wed, 16 Oct 2019 15:33:50 GMT
Connection: keep-alive
ETag: "5da7385e-9e0"
Accept-Ranges: bytes
- 查看缓存有效时长：
[root@alexis-01 ~]# curl -I http://www.apelearn.com/src/images/lesson_16.png
HTTP/1.1 200 OK
Server: nginx
Date: Fri, 18 Oct 2019 12:58:46 GMT
Content-Type: image/png
Content-Length: 5524
Connection: keep-alive
Last-Modified: Wed, 22 Nov 2017 09:10:54 GMT
ETag: "5a153f1e-1594"
Expires: Sun, 17 Nov 2019 12:58:46 GMT
Cache-Control: max-age=2592000
Accept-Ranges: bytes
max-age=2592000，单位是秒
也就是服务器上的过期时间为30天
- 设定过期时间
location ~* \.(png|jpeg|gif|js|css|bmp|flv)$
{
expires 1d;
access_log off;
}
- 设定完过期时间后，查看max-age
[root@alexis-01 ~]# curl -x127.0.0.1:80 -I http://bbs.ars4life.com/static/image/common/logo_88_31.gif
HTTP/1.1 200 OK
Server: nginx/1.16.1
Date: Fri, 18 Oct 2019 13:09:21 GMT
Content-Type: image/gif
Content-Length: 2528
Last-Modified: Wed, 16 Oct 2019 15:33:50 GMT
Connection: keep-alive
ETag: "5da7385e-9e0"
Expires: Sat, 19 Oct 2019 13:09:21 GMT
Cache-Control: max-age=86400
Accept-Ranges: bytes
设置完成后，过期时间为 24 小时
总结：
curl -x 用来指定目标服务器的IP和端口，例如：curl -x127.0.0.1:80 -I www.baidu.com
bc 是一个linux系统下的计算器，yum install -y bc
4.42 nginx防盗链
-
防盗链：
两个网站 A和 B，A 网站引用了 B 网站上的图片，这种行为就叫做盗链。
防盗链，就是要防止 A 应用 B 的图片 -
编辑虚拟主机配置文件，添加以下一段
location ~ \.(png|jpg|gif|jpeg|bmp|mp3|mp4|flv)$
{
valid_referers none blocked server_names *.test.com ;
if ($invalid_referer) {
return 403;
}
}
valid_referers 指的是白名单的 referer
none 表示空 referer，图片是打开的而不是跳转的
blocked 非法的链接，正常链接是 http:// 或 https:// 开头，如果不以他们开头的链接也可以成为合法的 referer，用的不多
server_names 就是上方定义的 server_name
*.test.com 就是白名单
1、准备： 先在 blog.ars4life.com 中上传一张图片，图片链接为 http://blog.ars4life.com/gakki.jpg
我们在bbs.ars4life.com 中发帖，并上传 http://blog.ars4life.com/gakki.jpg 图片到论坛中
此时我们还未进行防盗链配置，图片是可以正常显示的
- 2、下面将防盗链配置写入虚拟主机配置文件中
（可以在配置后增加关闭访问日志和设置静态文件过期时间等参数，access_log、expires）
location ~ \.(png|jpg|gif|jpeg|bmp|mp3|mp4|flv)$
{
valid_referers none blocked server_names www.ars4life.com ;
if ($invalid_referer) {
return 403;
}
access_log off;
expires 1d;
}
（以上字段只是防盗链部分，其余部分不在这里展示）
- 3、配置完成后，重新加载 nginx 配置
[root@alexis-01 blog.ars4life.com]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@alexis-01 blog.ars4life.com]# nginx -s reload
4、刷新页面，发现其他人已经无法查看到图片
5、防盗链成功！
补充：
rz 上传文件，yum install lrzsz
sz filename 这样去吧文件推送到win机器里
测试防盗链： curl -I -e “http://www.aaa.com/1.txt” http://www.ars4life.com/1.png
curl -e 指定自定义的 referer，只有白名单referer才会是200