使用AWSTATS自动分析Nginx日志
使用AWSTATS自动分析Nginx日志
环境说明：
Nginx安装目录：/usr/local/nginx
Nginx配置文件存放目录：/usr/local/nginx/conf/nginx.conf
Nginx日志目录：/usr/local/nginx/logs
Awstats安装目录：/usr/local/awstats
Awstats配置文件存放目录：/etc/awstats
日志切割脚本存放目录：/usr/local/nginx/sbin/cut_nginx_log.sh
一、Nginx日志切割
1、修改nginx.conf配置文件，使AWSTATS支持分析日志格式
# vim /usr/local/nginx/conf/nginx.conf
|
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
|
2、编辑Nginx日志切割脚本
# vim /usr/local/nginx/sbin/cut_nginx_log.sh
|
#!/bin/bash
mv /usr/local/nginx/logs/access.log /usr/local/nginx/logs/access_`date +%Y%m%d`.log
mv /usr/local/nginx/logs/error.log /usr/local/nginx/logs/error_`date +%Y%m%d`.log
killall -s USER1 nginx
|
# chmod +x /usr/local/nginx/sbin/cut_nginx_log.sh
3、
二、下载安装Awstats
1、下载安装
|
# wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gz
# tar -zxvf awstats-7.0.tar.gz
# mv awstats-7.0 /usr/local/awstats
# mkdir -p /var/lib/awstats
|
2、配置
|
# mkdir - p /tmp/src/wwwroot/cgi-bin/
# cp
/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf /tmp/src/wwwroot/cgi-bin/
# /usr/local/awstats/tools/awstats_configure.pl
-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/tmp/src
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ?
输入
y
|
回车
|
-----> Check for web server install
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
>
none 这里不是apache，写none，跳过#
|
回车
|
-----> Update model config file '/tmp/src/wwwroot/cgi-bin/awstats.model.conf'
File awstats.model.conf updated.
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ?
输入
y #创建一个全新的统计
|
回车
|
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
>
192.168.2.143被统计网站的域名 #
|
回车
|
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
> #回车继续
-----> Create config file '/etc/awstats/awstats.192.168.2.143.conf'
Config file /etc/awstats/awstats.192.168.2.143.conf created.
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/tmp/src/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143
Or if you have several config files and prefer having only one command:
/tmp/src/tools/awstats_updateall.pl now
Press ENTER to continue...
A SIMPLE config file has been created: /etc/awstats/awstats.192.168.2.143.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for '192.168.2.143' with command:
> perl awstats.pl -update -config=192.168.2.143
You can also build static report pages for '192.168.2.143' with command:
> perl awstats.pl -output=pagetype -config=192.168.2.143
Press ENTER to finish...
|
三、修改统计日志文件的路径
|
# vim /etc/awstats/awstats.192.168.2.143.conf -
LogFile="/var/log/httpd/mylog.log"修改成--->
LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-0.log"
|
这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，分析的执行顺序是：
Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志 –> 交由Awstats统计 –> 生成结果
四、执行Awstats 日志更新程序开始统计分析
|
#/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143
Create/Update database for config "/etc/awstats/awstats.192.168.2.143.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/nginx/logs/access_20121207.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 66466
Found 0 dropped records,
Found 0 comments,
Found 1 blank records,
Found 58472 corrupted records,
Found 0 old records,
Found 7993 new qualified records.
|
看到以上显示，证明日志切割和 Awstats 都已经运行无误了。统计分析完成后，结果还在 Awstats 的数据库中。在 Apache 上，可以直接打开 Perl 程序的网页查看统计。但本文开始时已经提到，Nginx 对 Perl 支持并不好，所以我们要换个方法，利用 awstats 的工具将统计的结果生成静态文件。
五、Awstats生成静态文件
|
# cp -rf /usr/local/awstats/wwwroot/ /usr/local/nginx/logs/
# mkdir /usr/local/nginx/logs/wwwroot/awstats
# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
|
/usr/local/awstats/tools/awstats_buildstaticpages.pl #Awstats 静态页面生成脚本
-update -config=192.168.2.143 #更新配置选项
-lang=cn #语音中文
-dir=/usr/local/nginx/logs/wwwroot/awstats/ #统计结果输出目录
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl #日志更新程序路径
六、修改nginx配置文件
|
# vim /usr/local/nginx/conf/nginx.conf
server {
listen 8080;
server_name localhost;
location ~ ^/awstats/ { # html 静态页面目录
root /usr/local/nginx/logs/wwwroot/awstats;
index index.html;
access_log off;
error_log off;
charset gb2312; }
location ~ ^/icon/ { # 图标目录
root /usr/local/nginx/logs/wwwroot;
index index.html;
access_log off;
error_log off;
charset gb2312;
}
}
|
七、配置Awstats自动运行
|
#
vim /usr/local/nginx/sbin/cut_nginx_log.sh
23 59 * * * root /usr/local/nginx/sbin/cut_nginx_log.sh
00 01 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
|
八、结果验证，URL输入 http://192.168.2.143:8080/awstats/awstats.192.168.2.143.html
