系统日志太多太分散的话就需要整合,并且分析,所以就有了这样一套东西,这样就大大的减轻了系统管理员的压力,不过现在这篇只是小试牛刀,很多应用功能还是没有用到,例如自定义日志收集过滤, 日志分析图表,等等,不过原理大致都基本如下,是可以举一反三的。
关于rsyslog和loganalyzer的配置简略架构流程图
一、rsyslog
【客户端】rsyslog配置
1.安装rsyslog
yum -y install rsyslog
2.配置rsyslog
/etc/rsyslog.conf
配置很多,但是只需要注意几个
# rsyslog v5 configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
$ModLoad imudp --注意这个
$UDPServerRun 514 --注意这个
# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
*.* @主rsyslog服务器ip或者host --注意这个
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
1.$ModLoad imudp和$UDPServerRun 514 开启rsyslog的日志远程传输,使用udp模式,当然也可以使用tcp模式,而且tcp也比udp更可靠,防止日志在传输过程丢失,只是需要建立稳定连接,消耗资源,各有所长,各有所需。
2.#.info;mail.none;authpriv.none;cron.none /var/log/messages 注释掉这个语句,改为.* @主rsyslog服务器ip或者host,这是为了配置rsyslog日志传输的目标,另外rsyslog的格式是分为2个方面的,一个是facitlity一个是priority,这个需要一点篇幅来说明,详细可以参阅科普时间或者官网,目前现在这个配置的意思是将所有级别的日志都传输到某个服务器。
3.客户端的rsyslog只需要配置这样就足够了。需要注意的是,如果开启了防火墙的话,那么记得514端口是rsyslog的传输端口,也要放开访问。
3.重启rsyslog服务
service rsyslog restart
【服务端】rsyslog配置
1.安装rsyslog
yum -y install rsyslog
2.配置rsyslog存储数据库(数据库的安装在下面那里一起写了)
yum -y install rsyslog-mysql
导入创库sql
cd /usr/share/doc/rsyslog-mysql-5.8.10/
[root@localhost rsyslog-mysql-5.8.10]# ls
createDB.sql
[root@localhost rsyslog-mysql-5.8.10]# mysql -u root -p < createDB.sql
这个sql会自动帮你创建rsyslog存储在mysql中的数据的表,等下可以直接被loganalyzer读取数据,然后使用。
连接数据库检查
mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.1.73 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| Syslog |
| mysql |
| test |
+--------------------+
4 rows in set (0.00 sec)
mysql> use Syslog
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+------------------------+
| Tables_in_Syslog |
+------------------------+
| SystemEvents |
| SystemEventsProperties |
+------------------------+
2 rows in set (0.00 sec)
配置rsyslog连接mysql账号和密码和授权
mysql> grant all on Syslog.* to 'rsysloga'@'localhost' identified by 'rsyslogp'; #设置用户访问数据库服务器中Syslog数据库的用户名和密码,因为rsyslog服务端和mysql数据库是在同一台机器上,所以只允许本机访问就可以了
Query OK, 0 rows affected (0.00 sec)
flush privileges; #刷新权限,及时生效
3.配置服务端的rsyslog.conf(数据库的安装在下面那里一起写了)
cat /etc/rsyslog.conf
# rsyslog v5 configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
$ModLoad ommysql --注意这个
# Provides UDP syslog reception
$ModLoad imudp --注意这个
$UDPServerRun 514 --注意这个
# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
*.* :ommysql:127.0.0.1,Syslog,rsysaloga,rsyslogp --注意这个
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
1.增加了$ModLoad ommysql这个模块,这个就是rsyslog连接mysql使用的模块
2.. :ommysql:127.0.0.1,Syslog,rsysaloga,rsyslogp 这里意思是说使用某个可以连接mysql的账号和密码,连接mysql,将数据传输到mysql数据库里面去。
3.udp传输配置依然要开启,那是因为rsyslog客户端会将日志以udp的方式传输到rsyslog服务端,所以双方都要开启同样的传输方式才可以完成传输。
4.需要注意的是,如果开启了防火墙的话,那么记得514端口是rsyslog的传输端口,也要放开访问。
4.重启rsyslog服务
service rsyslog restart
至此,rsyslog部分已经完成配置,若要检查是否配置成功,可以检查数据库里是否有数据即可。
mysql> select * from SystemEvents;
二、loganalyzer
可以理解为loganalyzer其实就是一个web平台来展现日志数据的而已。
1.下载并安装http+php+mysql套件
yum -y install httpd php php-mysql php-gd mysql mysql-server
httpd用来提供web服务
php使apache支持php,因为loganalyzer是用php编写
php-mysql用于loganalyzer连接数据库
php-gd用于绘图
mysql 是loganalyzer存储数据的地方
设置MySQL的root用户设置密码,因为MySQL被安装时,它的root用户时没有设置密码的,所以可以直接连或者设置一个,但是不影响我们这次配置任务。mysql需要启动,这个需要注意。
2.配置apache+php,并启动apache和mysql
因为yum安装的关系,所有一切都已经配置好了
如:grep -E 'Document|Listen' /etc/httpd/conf/httpd.conf |grep -v '^#'
Listen 80
DocumentRoot "/var/www/html"
如:grep -v '^#' /etc/httpd/conf.d/php.conf
LoadModule php5_module modules/libphp5.so
LoadModule php5_module modules/libphp5-zts.so
AddHandler php5-script .php
AddType text/html .php
DirectoryIndex index.php
启动httpd
service httpd start
启动mysql
service mysqld start
3.下载loganalyzer
下载地址:http://download.adiscon.com/loganalyzer/loganalyzer-3.6.6.tar.gz
将其放置到配置好的apache的web目录里面/var/www/html
tar -zxpf loganalyzer-3.6.6.tar.gz -C /var/www/html
cd /var/www/html/
[root@localhost html]# ls
loganalyzer-3.6.6
授权目录
chown -R apache.apache /var/www/html/loganalyzer-3.6.6/
4.配置loganalyzer
创建loganalyzer数据库和访问账号密码和授权
mysql> create database loganalyzer;
Query OK, 1 row affected (0.04 sec)
mysql> grant all on loganalyzer.* to loga@'localhost' identified by 'logp';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
生成config.php
cd /var/www/html/loganalyzer-3.6.6/contrib
chmod +x *
./configure.sh
在当前目录会生成config.php文件,然后将其放置到src目录去
cp config.php /var/www/html/loganalyzer-3.6.6/src/
在浏览器访问这台主机的80端口
http://服务端的ip/loganalyzer-3.6.6/src/install.php
备注:
1.step 2 会检查config.php的写入权限,如果没有请授权一下, chmod +w config.php
2.step 3 选择enable user database,使用自定义数据库,然后填写数据库访问信息,这里的数据库是指loganalyzer的用户数据库,而不是rsyslog日志存储的数据库,这里是需要注意的。并且选取require user to be login。
3.step 5 会将loganalyzer的相关用户表写入到数据库,可以检查loganalyzer的数据库就可以看到了。
4.step 6 配置loganalyzer的管理员账号,登录loganalyzer界面使用的。
5.step 7 是配置rsyslog的日志存储数据库的访问方法,在source type选择 mysql native,然后填写mysql的访问信息,记住,这里是rsyslog的日志存储数据库,不是loganalyzer的用户数据库。
6.完成后会自动跳转提示登录,登陆后就可以看到数据了。
三、科普时间
1.关于rsyslog的日志规则facitlity和priority
###rsyslog.conf中日志规则的定义的格式
facitlity.priority Target
#facility: 日志设备(可以理解为日志类型):
==============================================================
auth #pam产生的日志,认证日志
authpriv #ssh,ftp等登录信息的验证信息,认证授权认证
cron #时间任务相关
kern #内核
lpr #打印
mail #邮件
mark(syslog) #rsyslog服务内部的信息,时间标识
news #新闻组
user #用户程序产生的相关信息
uucp #unix to unix copy, unix主机之间相关的通讯
local 1~7 #自定义的日志设备
===============================================================
#priority: 级别日志级别:
=====================================================================
debug #有调式信息的,日志信息最多
info #一般信息的日志,最常用
notice #最具有重要性的普通条件的信息
warning, warn #警告级别
err, error #错误级别,阻止某个功能或者模块不能正常工作的信息
crit #严重级别,阻止整个系统或者整个软件不能正常工作的信息
alert #需要立刻修改的信息
emerg, panic #内核崩溃等严重信息
###从上到下,级别从低到高,记录的信息越来越少,如果设置的日志内性为err,则日志不会记录比err级别低的日志,只会记录比err更高级别的日志,也包括err本身的日志。
=====================================================================
Target:
#文件, 如/var/log/messages
#用户, root,*(表示所有用户)
#日志服务器,@172.16.22.1
#管道 | COMMAND
2.如果日志数量太大,内容太多,可以进行过滤记录日志
简单的方法可以在rsyslog客户端上的配置
:msg, !contains, "informational"
*.* @主rsyslog服务器ip或者host
在传输配置的上一行增加一个过滤配置,格式是严格的,一定要在上一行增加过滤配置,这里的意思是日志内容出现informational的就不记录。详细的过滤方式在官网上有说,需要的话就要慢慢按照他的方式来使用。
参考文档:
1.http://litaotao.blog.51cto.com/6224470/1283871
2.http://ftp.ics.uci.edu/pub/centos0/ics-custom-build/BUILD/rsyslog-3.19...
3.http://www.cnblogs.com/tobeseeker/archive/2013/03/10/2953250.html
原文链接:
http://www.godblessyuan.com/2015/05/02/rsyslog_loganalyzer_setting/