日志分割
-
随着网站的访问量增加，默认情况下
Apache的单个日志文件也会越来越大- 日志文件占用磁盘空间很大
- 查看相关信息不方便
- 对日志文件进行分割
- Apache自带
rotatelogs分割工具实现 - 第三方工具
cronolog分割
- Apache自带
rotatelogs分割
- 配置网站的日志文件转交给
rotatelogs分割处理 - 配置格式为
ErrorLog 4| rotatelogs命令的绝对路径-I日志文件路径/网站名-error %Y%m%d.log 86400"-
CustomLog "| rotatelogs命令路径-I日志文件路径/网站名-access %Y%m%d.log 86400" combined配合实例
[root@localhost ~]# yum install httpd -y //安装http服务
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
* extras: centos.ustc.edu.cn
...//省略部分内容...
已安装:
httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
完毕！
[root@localhost bin]# cd /usr/sbin/ //进入sbin目录
[root@localhost sbin]# ls rotat* //查看目录下是否生成命令
rotatelogs //安装http服务后生成rotatelogs命令
[root@localhost sbin]# cd /var/log/httpd/ //进入http服务日志存放目录
[root@localhost httpd]# ls //查看是否生成日志文件
[root@localhost httpd]# //安装完成后并没有生成目录文件
[root@localhost httpd]# vim /etc/httpd/conf/httpd.conf //进入http配置文件，进行常规配置
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80 //修改IPv4为本地地址
#Listen 80 //注释掉IPv6监听
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80 //开启域名服务
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
:wq
[root@localhost httpd]# systemctl stop firewalld.service //关闭防火墙
[root@localhost httpd]# setenforce 0 //关闭增强型安全功能
[root@localhost httpd]# systemctl start httpd.service //启动http服务
[root@localhost httpd]# ls //查看是否生成日志文件
access_log error_log //服务开启后成功生成日志文件
[root@localhost httpd]# vim conf/httpd.conf //进入更改主配置文件，做日志分割
...//省略部分内容...
# container, that host's errors will be logged there and not here.
#
#ErrorLog "logs/error_log" //复制此条目，并将此条目在下方粘贴
ErrorLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.error_%Y%m%log 86400"
//更改条目，引用rotatelogs命令，创建分割日志条目名称，每86400秒（一天）创建一次文件
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分内容...
# (Combined Logfile Format) you can use the following directive.
#
#CustomLog "logs/access_log" combined //复制此条目，并将此条目在下方粘贴
CustomLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.access_%Y%m%dlog 86400" combined
//更改条目，引用rotatelogs命令，创建分割日志条目名称，每86400秒（一天）创建一次文件
...//省略部分内容...
:wq
[root@localhost httpd]# systemctl stop httpd.service //停止http服务
[root@localhost httpd]# systemctl start httpd.service //启动http服务
[root@localhost httpd]# cd ..
[root@localhost log]# cd ~
[root@localhost ~]# cd /var/log/httpd/ //重新进入站点文件
[root@localhost httpd]# ls
access_log error_log www.kgc.com.error_20191105log //显示分割除来的日志文件
[root@localhost httpd]# date //查看时间
2019年 11月 05日 星期二 19:27:50 CST
[root@localhost httpd]# date -s 11/06/19 //将时间设置后一天
2019年 11月 06日 星期三 00:00:00 CST
[root@localhost httpd]# systemctl stop httpd.service //关闭http服务
[root@localhost httpd]# systemctl start httpd.service //启动服务
[root@localhost httpd]# cd ~ //回到家目录
[root@localhost ~]# cd /var/log/httpd/ //重新今日日志存放目录
[root@localhost httpd]# ls //查看
access_log error_log www.kgc.com.error_20191105log www.kgc.com.error_20191106log
//生成新的日志文件 第三方工具cronolog
- 源码编译安装
cronolog工具 cronolog安装包下载 - 配置网站日志文件转交给
cronolog分割处理 - 配置格式为:
ErrorLog "| cronolog命令的绝对路径日志文件路径/网站名-error_ %Y%m%d.log'
CustomLog "| cronolog命令的绝对路径|日志文件路径/网站名%Y%m%d.log" combined配置实例
重新开启一台Linux虚拟机，重新安装http服务
[root@localhost ~]# yum install httpd -y //安装服务
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
...//省略部分内容...
已安装:
httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
[root@localhost ~]# mount.cifs //192.168.100.10/lamp-c7 /mnt //远程挂载目录
Password for root@//192.168.100.10/lamp-c7:
[root@localhost ~]# cd /mnt/ //进入挂载目录
[root@localhost mnt]# ls //查看是否成功挂载
apr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt nginx-1.12.0.tar.gz
awstats-7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2
[root@localhost mnt]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm //安装软件包
警告：cronolog-1.6.2-14.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID 352c64e5: NOKEY
准备中... ################################# [100%]
正在升级/安装...
1:cronolog-1.6.2-14.el7 ################################# [100%]
[root@localhost mnt]# cd /usr/sbin/ //进入目录
[root@localhost sbin]# ls cronolog* //查看是否生成cronolog命令
cronolog //成功安装
[root@localhost sbin]# vim /etc/httpd/conf/httpd.conf //进入编辑配置文件
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80 //设置监听地址
#Listen 80
#
# Dynamic Shared Object (DSO) Support
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80 //设置域名
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
# container, that host's errors will be logged there and not here.
#
ErrorLog "| /usr/sbin/cronolog logs/www.kgc.com.error_%Y%m%dlog"
//调用cronolog命令创建错误日志文件
#
# LogLevel: Control the number of messages logged to the error_log.
...//省略部分内容...
# (Combined Logfile Format) you can use the following directive.
#
CustomLog "| /usr/sbin/cronolog logs/www.kgc.com.access_%Y%m%dlog" combined
//调用cronolog命令创建访问日志文件
...//省略部分内容...
:wq
[root@localhost sbin]# systemctl stop firewalld.service //关闭防火墙
[root@localhost sbin]# setenforce 0 //关闭增强性安全功能
[root@localhost sbin]# systemctl start httpd.service //开启http服务
[root@localhost sbin]# cd /var/log/httpd/ //进入日志文件目录
[root@localhost httpd]# ls //查看
www.kgc.com.error_20191105log //生成日志分割文件
[root@localhost httpd]# date //查看当前时间
2019年 11月 05日 星期二 20:11:57 CST
[root@localhost httpd]# date -s 11/06/19 //将时间设置为后一天
2019年 11月 06日 星期三 00:00:00 CST
[root@localhost httpd]# systemctl stop httpd.service //停止http服务
[root@localhost httpd]# systemctl start httpd.service //启动http服务
[root@localhost httpd]# cd ~ //回到家目录
[root@localhost ~]# cd /var/log/httpd/ //重新进入日志目录
[root@localhost httpd]# ls //查看
www.kgc.com.error_20191105log www.kgc.com.error_20191106log //生成新的日志文件AWStats日志分析系统介绍
AWStats日志分析系统
Perl语言开发的一款开源日志分析系统- 可用来分析
Apache、Samba、 Vsftpd、 IIS等服务 器的访问日志 - 信息结合
crond等计划任务服务，可对日志内容定期进行分析
部署AWStats日志分析系统
- 重新开启一台Linux虚拟机，安装http、DNS服务，并进行常规配置
[root@localhost ~]# yum install bind httpd -y
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.163.com
...//省略部分内容...
已安装:
bind.x86_64 32:9.11.4-9.P2.el7 httpd.x86_64 0:2.4.6-90.el7.centos
...//省略部分内容...
[root@localhost ~]# vim /etc/named.conf
...//省略部分内容...
options {
listen-on port 53 { any; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
recursing-file "/var/named/data/named.recursing";
secroots-file "/var/named/data/named.secroots";
allow-query { any; };
...//省略部分内容...
[root@localhost ~]# vim /etc/named.rfc1912.zones
...//省略部分内容...
zone "kgc.com" IN {
type master;
file "kgc.com.zone";
allow-update { none; };
};
...//省略部分内容...
[root@localhost named]# cp -p named.localhost kgc.com.zone
[root@localhost named]# vim kgc.com.zone
$TTL 1D
@ IN SOA @ rname.invalid. (
0 ; serial
1D ; refresh
1H ; retry
1W ; expire
3H ) ; minimum
NS @
A 127.0.0.1
www IN A 192.168.144.133
:wq
[root@localhost named]# vim /etc/httpd/conf/httpd.conf
...//省略部分内容...
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 192.168.144.133:80
#Listen 80
#
# Dynamic Shared Object (DSO) Support
...//省略部分内容...
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName www.kgc.com:80
#
# Deny access to the entirety of your server's filesystem. You must
...//省略部分内容...
:wq
[root@localhost named]# systemctl stop firewalld.service
[root@localhost named]# setenforce 0
[root@localhost named]# systemctl start named
[root@localhost named]# systemctl start httpd.service - 在客户机中测试网站是否可以访问
- 安装
AWStats软件包
[root@localhost named]# cd /etc/httpd/
[root@localhost httpd]# mount.cifs //192.168.100.10/lamp-c7 /mnt //将软件存放目录挂载在mnt目录下
Password for root@//192.168.100.10/lamp-c7: //直接回车
[root@localhost httpd]# cd /mnt/ //查看挂目录
[root@localhost mnt]# ls
apr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gz
apr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt nginx-1.12.0.tar.gz
awstats-7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2
[root@localhost mnt]# tar zxvf awstats-7.6.tar.gz -C /opt/ //解压awstats工具在opt目录
awstats-7.6/
awstats-7.6/tools/
awstats-7.6/tools/awstats_buildstaticpages.pl
...//省略部分内容...
[root@localhost mnt]# cd /opt/ //进入opt目录
[root@localhost opt]# ls
awstats-7.6 rh
[root@localhost opt]# mv awstats-7.6/ /usr/local/awstats //将解压的文件移动至/usr/local/目录下
[root@localhost opt]# cd /usr/local/ //进入local目录
[root@localhost local]# ls
awstats bin etc games include lib lib64 libexec sbin share src
[root@localhost local]# cd awstats/ //进入awstats目录
[root@localhost awstats]# ls
docs README.md tools wwwroot
[root@localhost awstats]# cd tools/ //进入工具存放目录
[root@localhost tools]# ls //查看可执行工具
awstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xslt
awstats_configure.pl dolibarr logresolvemerge.pl urlaliasbuilder.pl
awstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin- 为要统计的站点建立配置文件
[root@localhost tools]# ./awstats_configure.pl //执行配置工具
----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
...//省略部分内容...
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf //输入http配置文件路径
...//省略部分内容...
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y //输入y确认
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.kgc.com //输入网页域名
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
> //直接回车，默认存放配置文件路径即可
...//省略部分内容...
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue...
A SIMPLE config file has been created: /etc/awstats/awstats.www.kgc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.kgc.com' with command:
> perl awstats.pl -update -config=www.kgc.com
You can also read your statistics for 'www.kgc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.kgc.com //给出分析访问网址
Press ENTER to finish...
[root@localhost tools]# cd /etc/httpd/conf //进入http配置文件目录
[root@localhost conf]# vim httpd.conf //编辑配置文件
...//省略部分内容...
# This is to permit URL access to scripts/files in AWStats directory.
#
//自动生成配置文件
Options None
AllowOverride None
# Order allow,deny //注释掉访问权限设置
# Allow from all
Require all granted //添加条目分为所有人都可以访问
:wq- 修改站点统计配置文件
[root@localhost etc]# cd /etc/awstats/ //进入配置awstats配置文件目录
[root@localhost awstats]# ls //查看
awstats.www.kgc.com.conf //awstats配置文件
[root@localhost awstats]# ls /var/log/httpd/ //查看http服务日志文件名称
access_log error_log
[root@localhost awstats]# vim awstats.www.kgc.com.conf //进入awstats配置文件
...//省略部分内容...
# Example: "/pathtotools/logresolvemerge.pl *.log |"
#
LogFile="/var/log/httpd/access_log" //找到次条目，更改要分析的日志文件名称即可
# Enter the log file type you want to analyze.
...//省略部分内容...
# Example: "C:/awstats_data_dir"
# Default: "." (means same directory as awstats.pl)
#
DirData="/var/lib/awstats" //找到此条目，查看分析后数据的存放位置
# Relative or absolute web URL of your awstats cgi-bin directory.
...//省略部分内容...
:wq
[root@localhost awstats]# cd /var/lib/ //进入lib目录
[root@localhost lib]# mkdir awstats //创建awstats分析数据存放目录
[root@localhost lib]# ls //查看目录是否成功创建
AccountsService chrony flatpak initramfs misc PackageKit rpm-state systemd yum
alsa color fprint ipa-client mlocate plymouth rsyslog tpm
alternatives colord games iscsi net-snmp polkit-1 samba tuned
authconfig dav gdm libvirt NetworkManager postfix selinux udisks2
awstats dbus geoclue lldpad nfs pulse setroubleshoot upower
bluetooth dhclient gssproxy logrotate ntp rpcbind sss vmware
certmonger dnsmasq hyperv machines os-prober rpm stateless xkb
[root@localhost lib]# systemctl restart httpd.service //重启http服务-
在客户机中测试
awstats工具
- 这个时候访问的
awstats是没有做统计分析的，我们需要执行更新数据工具来执行统计
[root@localhost lib]# cd /usr/local/awstats/ //进入awstats工具目录
[root@localhost awstats]# ls
docs README.md tools wwwroot
[root@localhost awstats]# cd tools/ //进入工具目录
[root@localhost tools]# ls
awstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xslt
awstats_configure.pl dolibarr logresolvemerge.pl urlaliasbuilder.pl
awstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin
[root@localhost tools]# ./awstats_updateall.pl now //执行工具进行更新，注意不要忘记后面加now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.kgc.com -configdir="/etc/awstats"' to update config www.kgc.com
Create/Update database for config "/etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 48
Found 0 dropped records,
Found 0 comments,
Found 0 blank records,
Found 0 corrupted records, //成功更新
Found 0 old records,
Found 48 new qualified records.- 在客户机重新访问
awstats
- 设置周期性计划任务，自动统计访问信息，并优化访问路径
[root@localhost tools]# crontab -e //设置周期性计划任务
*/5 * * * * /usr/local/awstats/tools/./awstats_updateall.pl now //设置每5分钟执行一次
~
~
:wq
[root@localhost tools]# systemctl start crond //启动周期性计划任务
[root@localhost tools]# cd /var/www/html/ //进入http站点目录
[root@localhost html]# vim aws.html //编辑网页
//编辑网页内容
:wq //保存退出- 在客户机中测试优化的访问路径是否可以使用
