一.简述
AWStats 软件是一个免费的强大的基于Perl的WEB日志分析工具，显示你所有的网页/邮件/ FTP统计包括访问，访问者，页面，点击，高峰时间，操作系统，浏览器，搜索引擎，关键字，机器人访问，断开的链接和更多的阻力截图排序。
（1）功能:它可以统计您站点的如下信息：
一：访问量，访问次数，页面浏览量，点击数，数据流量等
二：精确到每月、每日、每小时的数据
三：访问者国家
四：访问者IP
五：Robots/Spiders的统计
六：访客持续时间
七：对不同Files type 的统计信息
八：Pages-URL的统计
九：访客操作系统浏览器等信息
十：其它信息（搜索关键字等等）
-----百度百科
（2）工作原理
工作原理
AWStats的功能很多，我在此主要用它来分析apache服务器的日志。安装使用之前还是说说大致的工作原理，AWStats提供一系列的perl脚本实现：服务配置，日志读取，报表生成等功能。而功能实现的具体执行过程是：首先，当然是apache将访问情况记录到日志中，AWStats每次执行更新时读取这些日志，分析日志数据，将结果存储到数据库中，(这个数据库是AWStats自带的（就是一文本文件），并不需要第三方软件支持。)，最后AWStats提供一个cgi程序通过web页面来显示数据库中所统计的数据。
工作模式
AWStats的工作模式是这样的：
分析日志：运行后将这样的日志统计结果归档到一个AWStats的数据库（纯文本）里；
输出日志：分两种形式
一种是通过cgi程序读取统计结果数据库输出（Linux中）；
一种是运行后台脚本将输出导出成静态文件(Windows中)；
（3）nginx安装配置详解
一.配置nginx日志格式并重新加载nginx，修改nginx.conf的日志格式，不然awstats无法分析。
[root@Monitor server]# vim /usr/local/nginx/conf/nginx.conf
worker_processes 1;
user appuser appuser;
error_log /data/nginx/error.log;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /data/nginx/access.log;
[root@Monitor server]# /usr/local/nginx/sbin/nginx -s reload
####注意要点：日志格式里的双引号不能漏了且每个参数之间是一个空格分隔，因为只要有细微的差别，awstats就无法读取日志
二.安装Awstats。
Awstats的安装有两种方式，一种是yum install awstats。二种是去官方网（https://awstats.sourceforge.io）直接下载安装包来进行安装。本文主要的是通过下载安装包进行安装
（1）通过yum直接进行安装
[root@ittestserver1 opt]# yum install -y awstats Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile epel/metalink | 6.0 kB 00:00 * base: mirrors.163.com * epel: mirror.premi.st * extras: mirrors.zju.edu.cn * updates: mirrors.163.com base | 3.7 kB 00:00 epel | 4.3 kB 00:00 epel/primary_db | 5.9 MB 00:01 extras | 3.4 kB 00:00 jenkins | 2.9 kB 00:00 jenkins/primary_db | 21 kB 00:00 updates | 3.4 kB 00:00 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package awstats.noarch 0:7.0-3.el6 will be installed --> Processing Dependency: perl(URI::Escape) for package: awstats-7.0-3.el6.noarch --> Processing Dependency: perl(LWP::UserAgent) for package: awstats-7.0-3.el6.noarch --> Processing Dependency: perl(HTML::Entities) for package: awstats-7.0-3.el6.noarch --> Running transaction check ---> Package perl-HTML-Parser.x86_64 0:3.64-2.el6 will be installed --> Processing Dependency: perl(HTML::Tagset) >= 3.03 for package: perl-HTML-Parser-3.64-2.el6.x86_64 --> Processing Dependency: perl(HTML::Tagset) for package: perl-HTML-Parser-3.64-2.el6.x86_64 ---> Package perl-URI.noarch 0:1.40-2.el6 wil ---> Package perl-libs.x86_64 4:5.10.1-141.el6_7.1 will be updated ---> Package perl-libs.x86_64 4:5.10.1-144.el6 will be an update ---> Package perl-version.x86_64 3:0.77-141.el6_7.1 will be updated ---> Package perl-version.x86_64 3:0.77-144.el6 will be an update --> Finished Dependency Resolution Verifying : 3:perl-version-0.77-141.el6_7.1.x86_64 25/29 Verifying : 4:perl-libs-5.10.1-141.el6_7.1.x86_64 26/29 Verifying : perl-ExtUtils-MakeMaker-6.55-141.el6_7.1.x86_64 27/29 Verifying : 1:perl-ExtUtils-ParseXS-2.2003.0-141.el6_7.1.x86_64 28/29 Verifying : perl-Test-Harness-3.17-141.el6_7.1.x86_64 29/29 Installed: awstats.noarch 0:7.0-3.el6 Dependency Installed: perl-Compress-Raw-Zlib.x86_64 1:2.021-144.el6 perl-Compress-Zlib.x86_64 0:2.021-144.el6 perl-HTML-Parser.x86_64 0:3.64-2.el6 perl-HTML-Tagset.noarch 0:3.20-4.el6 perl-IO-Compress-Base.x86_64 0:2.021-144.el6 perl-IO-Compress-Zlib.x86_64 0:2.021-144.el6 perl-URI.noarch 0:1.40-2.el6 perl-libwww-perl.noarch 0:5.833-5.el6 Dependency Updated: perl.x86_64 4:5.10.1-144.el6 perl-ExtUtils-MakeMaker.x86_64 0:6.55-144.el6 perl-ExtUtils-ParseXS.x86_64 1:2.2003.0-144.el6 perl-Module-Pluggable.x86_64 1:3.90-144.el6 perl-Pod-Escapes.x86_64 1:1.04-144.el6 perl-Pod-Simple.x86_64 1:3.13-144.el6 perl-Test-Harness.x86_64 0:3.17-144.el6 perl-devel.x86_64 4:5.10.1-144.el6 perl-libs.x86_64 4:5.10.1-144.el6 perl-version.x86_64 3:0.77-144.el6 Complete!
（2）通过下载安装包进行安装。下载地址为：https://prdownloads.sourceforge.net/awstats/awstats-7.6.tar.gz
[root@Monitor soft]# wget https://prdownloads.sourceforge.net/awstats/awstats-7.6.tar.gz --2017-09-18 11:17:28-- https://prdownloads.sourceforge.net/awstats/awstats-7.6.tar.gz 正在解析主机 prdownloads.sourceforge.net... 216.34.181.59 正在连接 prdownloads.sourceforge.net|216.34.181.59|:443... 已连接。 已发出 HTTP 请求，正在等待回应... 301 Moved Permanently 位置：https://downloads.sourceforge.net/project/awstats/AWStats/7.6/awstats-7.6.tar.gz [跟随至新的 URL] --2017-09-18 11:17:29-- https://downloads.sourceforge.net/project/awstats/AWStats/7.6/awstats-7.6.tar.gz 正在解析主机 downloads.sourceforge.net... 216.34.181.59 正在连接 downloads.sourceforge.net|216.34.181.59|:443... 已连接。 已发出 HTTP 请求，正在等待回应... 302 Found 位置：https://nchc.dl.sourceforge.net/project/awstats/AWStats/7.6/awstats-7.6.tar.gz [跟随至新的 URL] --2017-09-18 11:17:42-- https://nchc.dl.sourceforge.net/project/awstats/AWStats/7.6/awstats-7.6.tar.gz 正在解析主机 nchc.dl.sourceforge.net... 211.79.60.17, 2001:e10:ffff:1f02::17 正在连接 nchc.dl.sourceforge.net|211.79.60.17|:443... 已连接。 已发出 HTTP 请求，正在等待回应... 200 OK 长度：2955876 (2.8M) [application/x-gzip] 正在保存至: “awstats-7.6.tar.gz” 5% [==> ] 163,840 53.1K/s eta(英国中部时 6% [===> ] 196,608 56.5K/s eta(英国中部时 7% [===> ] 229,376 59.4K/s eta(英国中部时 8% [====> ] 245,760 57.8K/s eta(英国中部时 9% [=====> ] 278,528 55.4K/s eta(英国中部时10% [=====> ] 311,296 57.5K/s eta(英国中部时11% [======> ] 327,680 56.2K/s eta(英国中部时11% [======> ] 344,064 56.1K/s eta(英国中部时12% [======> ] 360,448 55.6K/s eta(英国中部时12% [=======> ] 376,832 55.1K/s eta(英国中部时13% [=======> ] 393,216 54.7K/s eta(英国中部时14% [========> ] 425,984 56.1K/s eta(英国中部时14% [========> ] 442,368 55.6K/s eta(英国中部时15% [========> ] 458,752 55.8K/s eta(英国中部时16% [=========> ] 475,136 56.2K/s eta(英国中部时16% [=========> ] 491,520 54.4K/s eta(英国中部时17% [=========> ] 507,904 54.8K/s eta(英国中部时17% [==========> ] 524,288 53.0K/s eta(英国中部时18% [===========> ] 557,056 55.4K/s eta(英国中部时19% [===========> ] 573,440 55.8K/s eta(英国中部时20% [============> ] 606,208 58.2K/s eta(英国中部时21% [============> ] 622,592 56.3K/s eta(英国中部时22% [=============> ] 655,360 56.4K/s eta(英国中部时23% [==============> ] 704,512 60.8K/s eta(英国中部时25% [===============> ] 753,664 66.4K/s eta(英国中部时27% [================> ] 802,816 68.7K/s eta(英国中部时29% [==================> ] 884,736 78.2K/s eta(英国中部时32% [===================> ] 966,656 86.6K/s eta(英国中部时36% [======================> ] 1,064,960 97.5K/s eta(英国中部时39% [========================> ] 1,179,648 111K/s eta(英国中部时42% [=========================> ] 1,245,184 117K/s eta(英国中部时48% [=============================> ] 1,425,408 135K/s eta(英国中部时52% [================================> ] 1,540,096 150K/s eta(英国中部时54% [==================================> ] 1,622,016 159K/s eta(英国中部时57% [===================================> ] 1,703,936 166K/s eta(英国中部时60% [=====================================> ] 1,785,856 174K/s eta(英国中部时62% [=======================================> ] 1,851,392 177K/s eta(英国中部时63% [=======================================> ] 1,867,776 178K/s eta(英国中部时65% [=========================================> ] 1,949,696 183K/s eta(英国中部时69% [===========================================> ] 2,048,000 193K/s eta(英国中部时70% [============================================> ] 2,097,152 194K/s eta(英国中部时72% [=============================================> ] 2,146,304 197K/s eta(英国中部时74% [==============================================> ] 2,195,456 199K/s eta(英国中部时75% [===============================================> ] 2,244,608 199K/s eta(英国中部时77% [================================================> ] 2,293,760 199K/s eta(英国中部时79% [=================================================> ] 2,342,912 199K/s eta(英国中部时80% [==================================================> ] 2,392,064 195K/s eta(英国中部时82% [===================================================> ] 2,441,216 187K/s eta(英国中部时84% [====================================================> ] 2,490,368 184K/s eta(英国中部时85% [=====================================================> ] 2,539,520 167K/s eta(英国中部时88% [=======================================================> ] 2,605,056 167K/s eta(英国中部时89% [========================================================> ] 2,637,824 142K/s eta(英国中部时90% [========================================================> ] 2,670,592 131K/s eta(英国中部时91% [=========================================================> ] 2,703,360 124K/s eta(英国中部时92% [==========================================================> ] 2,736,128 125K/s eta(英国中部时93% [==========================================================> ] 2,768,896 118K/s eta(英国中部时94% [===========================================================> ] 2,801,664 109K/s eta(英国中部时95% [============================================================> ] 2,834,432 107K/s eta(英国中部时97% [=============================================================> ] 2,867,200 105K/s eta(英国中部时98% [=============================================================> ] 2,899,968 102K/s eta(英国中部时99% [==============================================================> ] 2,932,736 100K/s eta(英国中部时100%[===============================================================>] 2,955,876 102K/s in 26s 2017-09-18 11:18:09 (112 KB/s) - 已保存 “awstats-7.6.tar.gz” [2955876/2955876]) [root@Monitor soft]# rm -rf /usr/local/awstats/ [root@Monitor soft]# tar xf awstats-7.6.tar.gz [root@Monitor soft]# mv awstats-7.6 awstats-7.6/ awstats-7.6.tar.gz [root@Monitor soft]# mv awstats-7.6 /usr/local/ [root@Monitor awstats]# mkdir /data/awstats
三.配置Awstats
1,进入/usr/local/awstats/tools/目录，执行配置脚本awstats_configure.pl：
[root@Monitor awstats]# cd /usr/local/awstats/tools/
[root@Monitor tools]# ./awstats_configure.pl
----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).
-----> Running OS detected: Linux, BSD or Unix
-----> Check for web server install
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> none #####跳过服务的设置
Your web server config file(s) could not be found.
You will need to setup your web server manually to declare AWStats
script as a CGI, if you want to build reports dynamically.
See AWStats setup documentation (file docs/index.html)
-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
File awstats.model.conf updated.
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y #####点击y创建一个新的配置文件
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.gjb.com #####输入你网站的域名
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
> #####直接回车键
-----> Create config file '/etc/awstats/awstats.www.gjb.com.conf'
Config file /etc/awstats/awstats.www.gjb.com.conf created. ##配置文件所在路径
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.gjb.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue... #####直接按回车键
A SIMPLE config file has been created: /etc/awstats/awstats.www.gjb.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.gjb.com' with command:
> perl awstats.pl -update -config=www.gjb.com
You can also build static report pages for 'www.gjb.com' with command:
> perl awstats.pl -output=pagetype -config=www.gjb.com
Press ENTER to finish... #####直接按回车键
2.编辑日志配置文件，修改LogFile参数路径为你nginx日志文件的路径
[root@Monitor tools]# vim /etc/awstats/awstats.www.gjb.com.conf # AWSTATS CONFIGURE FILE 7.3 #----------------------------------------------------------------------------- # Copy this file into awstats.www.mydomain.conf and edit this new config file # to setup AWStats (See documentation in docs/ directory). # "LogFile" contains the web, ftp or mail server log file to analyze. # Possible values: A full path, or a relative path from awstats.pl directory. # Example: "/var/log/apache/access.log" # Example: "../logs/mycombinedlog.log" # You can also use tags in this filename if you need a dynamic file name # depending on date or time (Replacement is made by AWStats at the beginning # of its execution). This is available tags : # %YYYY-n is replaced with 4 digits year we were n hours ago # %YY-n is replaced with 2 digits year we were n hours ago LogFile="/data/nginx/0914.log-20170917" #日志文件存放路径 DirData="/var/lib/awstats" #创建生成的数据路径 Lang="cn" #默认语言中文 SkipHosts="127.0.0.1 REGEX[^192\.168\.]" #本地及内部的访问不做分析统计 LevelForWormsDetection=2 #日志等级，不对警告日志进行统计
四.执行awstats日志分析awstats.pl语句
[root@Monitor tools]# /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.gjb.com Create/Update database for config "/etc/awstats/awstats.www.gjb.com.conf" by AWStats version 7.6 (build 20161204) From data in log file "/data/nginx/0914.log-20170917"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... Flush history file on disk (unique hosts reach flush limit of 20000) Flush history file on disk (unique hosts reach flush limit of 20000) Flush history file on disk (unique hosts reach flush limit of 20000) Flush history file on disk (unique hosts reach flush limit of 20000) Flush history file on disk (unique hosts reach flush limit of 20000) Jumped lines in file: 0 Parsed lines in file: 3753576 Found 2 dropped records, Found 0 comments, Found 0 blank records, Found 0 corrupted records, Found 0 old records, Found 3753574 new qualified records. ##查看一下awstats数据库文件 [root@Monitor awstats]# du -sh * 8.3M awstats092017.www.gjb.com.txt
五.配置awstats生成静态页面
利用awstats的工具将统计的结果生成静态文件：
[root@Monitor awstats]# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.gjb.com -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -update -configdir= Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output Build alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=alldomains Build allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=allhosts Build lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=lasthosts Build unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=unknownip Build allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=allrobots Build lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=lastrobots Build session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=session Build urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=urldetail Build urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=urlentry Build urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=urlexit Build osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=osdetail Build unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=unknownos Build browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=browserdetail Build unknownbrowser page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=unknownbrowser Build downloads page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=downloads Build refererse page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=refererse Build refererpages page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=refererpages Build keyphrases page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=keyphrases Build keywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=keywords Build errors400 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=errors400 Build errors403 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=errors403 Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.gjb.com -staticlinks -lang=cn -output=errors404 23 files built. Main HTML page is 'awstats.www.gjb.com.html'.
如果想每天都更新的话设置定时任务：
[root@Monitor awstats]# crontab -e no crontab for root - using an empty one 1 0 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config= -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
六.配置nginx代理来查看生成的效果图。
[root@Monitor awstats]# vim /usr/local/nginx/conf/server/server.conf
server {
listen 8000;
server_name xn3.lqb.com;
root /data/;
location ~ ^/awstats {
# root /opt/www/awstats;
index index.html;
access_log /opt/www/awstats.log;
error_log /opt/www/error.log;
#charset utf-8;
charset gb2312;
#auth_basic "admin";
# auth_basic_user_file conf/admin.pass;
}
location ~ ^/icon/{
root /usr/local/awstats/wwwroot;
index index.html;
access_log off;
error_log off;
}
}
[root@Monitor awstats]# /usr/local/nginx/sbin/nginx -s reload
生成的效果图如下：
至此nginx日志分析完成