Nginx配置Awstats分析Nginx日志笔记
1、修改Nginx日志格式：
log_format json '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /data/nginx_logs/access.log json;
2、Nginx日志切割（shell脚本，略）
3、安装GeoIP库
yum -y install GeoIP GeoIP-devel perl-Geo-IP
4、安装Awstats
tar xvf awstats-7.4.tar.gz mv awstats-7.4 /usr/local/awstats cd /usr/local/ chown root:root -R awstats/ chmod +x /usr/local/awstats/tools/*.pl chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl
5、运行脚本生成配置
cd /usr/local/awstats/tools/ ./awstats_configure.pl
脚本交互1：
Config file path ('none' to skip web server setup):
因为在此我们使用的是nginx，所以填写none
脚本交互2：
Do you want me to build a new AWStats config/profile file (required if first install) [y/N] 输入Y创建一个新的统计配置文件。
脚本交互3：
Your web site, virtual server or profile name: > www.test.com 在这输入自己的网站域名
脚本交互4：
Directory path to store config file(s) (Enter for default): > 使用默认配置，生成配置文件
后面的直接按回车就可以
6、修改上面生成的配置文件/etc/awstats/awstats.app.mir.6wtx.com.conf
LogFile="/data/nginx_logs/cut_logs/%YYYY-24%MM-24%DD-24/ DirData="/data/awstats" LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat" LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/share/GeoIP/GeoLiteCity.dat"
7、生成数据文件并配置Nginx，此处应该有两种办法：
一种是用fastcgi调用perl分析数据文件
一种是直接生成静态文件nginx解析
第一种办法：
7.1.1、安装FCGI和FCGI::ProcManager
cpan>install FCGI cpan>install FCGI::ProcManager
7.1.2、创建fastcgi来执行perl：/usr/local/nginx/sbin/fcgi
#!/usr/bin/perl
use FCGI;
#perl -MCPAN -e 'install FCGI'
use Socket;
use POSIX qw(setsid);
#use Fcntl;
require 'syscall.ph';
&daemonize;
#this keeps the program alive or something after exec'ing perl scripts
END() { } BEGIN() { }
*CORE::GLOBAL::exit = sub { die "fakeexit\nrc=".shift()."\n"; };
eval q{exit};
if ($@) {
exit unless $@ =~ /^fakeexit/;
};
&main;
sub daemonize() {
chdir '/' or die "Can't chdir to /: $!";
defined(my $pid = fork) or die "Can't fork: $!";
exit if $pid;
setsid or die "Can't start a new session: $!";
umask 0;
}
sub main {
#$socket = FCGI::OpenSocket( "127.0.0.1:8999", 10 );
$socket = FCGI::OpenSocket( "/usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock", 10 );
#use UNIX sockets - user running this script must have w access to the 'nginx' folder!!
$request = FCGI::Request( \*STDIN, \*STDOUT, \*STDERR, \%req_params, $socket );
if ($request) { request_loop()};
FCGI::CloseSocket( $socket );
}
sub request_loop {
while( $request->Accept() >= 0 ) {
#processing any STDIN input from WebServer (for CGI-POST actions)
$stdin_passthrough ='';
$req_len = 0 + $req_params{'CONTENT_LENGTH'};
if (($req_params{'REQUEST_METHOD'} eq 'POST') && ($req_len != 0) ){
my $bytes_read = 0;
while ($bytes_read < $req_len) {
my $data = '';
my $bytes = read(STDIN, $data, ($req_len - $bytes_read));
last if ($bytes == 0 || !defined($bytes));
$stdin_passthrough .= $data;
$bytes_read += $bytes;
}
}
#running the cgi app
if ( (-x $req_params{SCRIPT_FILENAME}) && #can I execute this?
(-s $req_params{SCRIPT_FILENAME}) && #Is this file empty?
(-r $req_params{SCRIPT_FILENAME}) #can I read this file?
){
pipe(CHILD_RD, PARENT_WR);
my $pid = open(KID_TO_READ, "-|");
unless(defined($pid)) {
print("Content-type: text/plain\r\n\r\n");
print "Error: CGI app returned no output - Executing $req_params
{SCRIPT_FILENAME} failed !\n";
next;
}
if ($pid > 0) {
close(CHILD_RD);
print PARENT_WR $stdin_passthrough;
close(PARENT_WR);
while(my $s = <KID_TO_READ>) { print $s; }
close KID_TO_READ;
waitpid($pid, 0);
} else {
foreach $key ( keys %req_params){
$ENV{$key} = $req_params{$key};
}
# cd to the script's local directory
if ($req_params{SCRIPT_FILENAME} =~ /^(.*)\/[^\/]+$/) {
chdir $1;
}
close(PARENT_WR);
close(STDIN);
#fcntl(CHILD_RD, F_DUPFD, 0);
syscall(&SYS_dup2, fileno(CHILD_RD), 0);
#open(STDIN, "<&CHILD_RD");
exec($req_params{SCRIPT_FILENAME});
die("exec failed");
}
}
else {
print("Content-type: text/plain\r\n\r\n");
print "Error: No such CGI app - $req_params{SCRIPT_FILENAME} may not exist or is
not executable by this process.\n";
}
}
}
7.1.3、授权：
chmod 755 /usr/local/nginx/sbin/fcgi
7.1.4、启动：
perl /usr/local/nginx/sbin/fcgi >/dev/null 2>&1
7.1.5、授权socker让Nginx调用：
chown www:www /usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock
7.1.6、创建/usr/local/nginx/conf/fastcgi_params1
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_read_timeout 60;
7.1.7、Nginx增加虚拟主机：
server {
listen 33333;
server_name xxxxxxxxxxx;
location / {
root /data/awstats;
index index.html index.htm;
}
location ~* ^/cgi-bin/.*\.pl$ {
root /usr/local/awstats/wwwroot;
fastcgi_pass unix:/usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock;
fastcgi_index index.pl;
include fastcgi_params1;
charset gb2312;
}
location ~ ^/icon/ { # 图标目录
root /usr/local/awstats/wwwroot;
index index.html;
access_log off;
error_log off;
}
}
最后重启Nginx
7.1.8、生成awstats数据文件：
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.test.com
7.1.9、打开URL查看效果：http://xxxxxxxxx:33333/cgi-bin/awstats.pl?config=www.test.com
第二种办法：
7.2.1、配置Nginx，不需要调用perl
server {
listen 44444;
server_name xxxxxxxxxxx;
root /data/awstats;
index index.html;
access_log off;
error_log off;
charset gb2312;
location ~ ^/icon/ { # 图标目录
root /usr/local/awstats/wwwroot;
index index.html;
access_log off;
error_log off;
}
}
重启Nginx
7.2.2、生成awstats静态数据文件：
/usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.test.com -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
7.2.3、 打开URL查看效果：http://xxxxxxxxxx:44444/awstats.www.test.com.html
8、添加定时任务：
5 0 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.sunsky.com >/dev/null 2>&1 10 0 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.test.com -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl >/dev/null 2>&1