在Linux中,服务、内核输出的日志信息都由rsyslog服务收集、展现。
一、ryslog 分为两部分:
1. syslogd,用户手机应用程序产生的日志信息。
2. klogd , 用于收集内核启动时输出的信息,通常保存为二进制文件,可有dmesg命令查看。
二、rsyslog RPM包组成:
[auditor@node1 ~]$ rpm -ql rsyslog
/etc/logrotate.d/syslog
/etc/pki/rsyslog
/etc/rsyslog.conf
/etc/rsyslog.d
/etc/sysconfig/rsyslog
/usr/bin/rsyslog-recover-qi.pl
/usr/lib/systemd/system/rsyslog.service
/usr/lib64/rsyslog
/usr/lib64/rsyslog/imdiag.so
/usr/lib64/rsyslog/imfile.so
/usr/lib64/rsyslog/imjournal.so
/usr/lib64/rsyslog/imklog.so
/usr/lib64/rsyslog/immark.so
/usr/lib64/rsyslog/impstats.so
/usr/lib64/rsyslog/imptcp.so
/usr/lib64/rsyslog/imtcp.so
/usr/lib64/rsyslog/imudp.so
/usr/lib64/rsyslog/imuxsock.so
/usr/lib64/rsyslog/lmnet.so
/usr/lib64/rsyslog/lmnetstrms.so
/usr/lib64/rsyslog/lmnsd_ptcp.so
/usr/lib64/rsyslog/lmregexp.so
/usr/lib64/rsyslog/lmstrmsrv.so
/usr/lib64/rsyslog/lmtcpclt.so
/usr/lib64/rsyslog/lmtcpsrv.so
/usr/lib64/rsyslog/lmzlibw.so
/usr/lib64/rsyslog/mmanon.so
/usr/lib64/rsyslog/mmcount.so
/usr/lib64/rsyslog/mmutf8fix.so
/usr/lib64/rsyslog/omjournal.so
/usr/lib64/rsyslog/ommail.so
/usr/lib64/rsyslog/omprog.so
/usr/lib64/rsyslog/omruleset.so
/usr/lib64/rsyslog/omstdout.so
/usr/lib64/rsyslog/omtesting.so
/usr/lib64/rsyslog/omuxsock.so
/usr/lib64/rsyslog/pmaixforwardedfrom.so
/usr/lib64/rsyslog/pmcisconames.so
/usr/lib64/rsyslog/pmlastmsg.so
/usr/lib64/rsyslog/pmrfc3164sd.so
/usr/lib64/rsyslog/pmsnare.so
/usr/sbin/rsyslogd
/usr/share/doc/rsyslog-7.4.7
/usr/share/doc/rsyslog-7.4.7/AUTHORS
/usr/share/doc/rsyslog-7.4.7/COPYING
/usr/share/doc/rsyslog-7.4.7/COPYING.ASL20
/usr/share/doc/rsyslog-7.4.7/COPYING.LESSER
/usr/share/doc/rsyslog-7.4.7/ChangeLog
/usr/share/man/man5/rsyslog.conf.5.gz
/usr/share/man/man8/rsyslogd.8.gz
/var/lib/rsyslog
/etc/rsyslog.conf #配置文件
/usr/lib64/*.so #rsyslog提供的模块,其中im开头的用于收集日志,om开头的模块用于输出、存储日志
三、rsyslog 配置文件
/etc/rsyslog.conf
#### MODULES #### 用于加载模块
# Provides UDP syslog reception 定义UDP/514端口接收日志
#$ModLoad imudp
#$UDPServerRun 514
# Provides TCP syslog reception 定义TCP/514端口接收日志
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES #### 定义全局选项
#### RULES #### 定义收集服务、程序什么级别的日志以及存放何处
格式:
Facility.Priority Target
Facility: 定义设施,按功能对日志进行分类
a. Auth 认证相关类的日志
b. AuthPriv 认证、授权相关类的日志
c. cron 计划任务日志
d. daemon 守护进程类日志
e. local0-local7 允许用户自定义日志类
Priority:
debug 调试日志
info 信息日志
notice 通知日志
warn 警告日志
error 错误日志
crit 蓝色警报日志
alert 橙色警报日志
emerg 红色警报日志
Target:
@Host 将日志发送至某一主机
USER_NAME 将日志发送至某在线用户
/PATH/TO/SOMEFILE 将日志发送至某一文件路径,例如/var/log
ommysql,host,db_name,user,password 将日志发送至MySQL中存储
四、测试rsyslog
需求:
将node1作为rsyslog Server,接受node2发送过来的日志。
node1:192.168.80.10
node2:192.168.80.11
node1的配置:
#开启日志接受功能 UDP/514
[root@node1 ~]# vim /etc/rsyslog.conf
$ModLoad imudp
$UDPServerRun 514
[root@node1 ~]# systemctl restart rsyslog
[root@node1 ~]# ss -unl | grep 514
UNCONN 0 0 *:514 *:*
UNCONN 0 0 :::514 :::*
node2的配置:
[root@node2 ~]# vim /etc/rsyslog.conf
*.* @192.168.80.10:514
[root@node2 ~]# systemctl restart rsyslog
[root@node2 ~]# systemctl restart vsftpd
验证:其中有不少node2的vsftpd日志
[root@node1 ~]# tailf /var/log/messages
Jul 14 02:15:12 node2 systemd: Starting Vsftpd ftp daemon...
Jul 14 02:15:12 node2 systemd: Started Vsftpd ftp daemon.
Jul 14 02:15:46 node2 systemd: Stopping Vsftpd ftp daemon...
Jul 14 02:15:46 node2 systemd: Starting Vsftpd ftp daemon...
Jul 14 02:15:46 node2 systemd: Started Vsftpd ftp daemon.
Jul 14 02:15:51 node2 systemd: Starting System Logging Service...
Jul 14 02:15:51 node2 systemd: Started System Logging Service.
Jul 14 02:15:58 node2 systemd: Stopping Vsftpd ftp daemon...
Jul 14 02:15:58 node2 systemd: Starting Vsftpd ftp daemon...
Jul 14 02:15:58 node2 systemd: Started Vsftpd ftp daemon.
Jul 14 02:19:49 node2 kernel: perf: interrupt took too long (23735 > 23313), lowering kernel.perf_event_max_sample_rate to 8000
Jul 27 07:00:01 node1 systemd: Started Session 194 of user root.
Jul 27 07:00:01 node1 systemd: Starting Session 194 of user root.
Jul 27 07:01:01 node1 systemd: Started Session 195 of user root.
Jul 27 07:01:01 node1 systemd: Starting Session 195 of user root.
Jul 14 02:21:08 node2 systemd: Starting Cleanup of Temporary Directories...
Jul 14 02:21:08 node2 systemd: Started Cleanup of Temporary Directories.
Jul 14 02:21:37 node2 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6564" x-info="http://www.rsyslog.com"] exiting on signal 15.
Jul 14 02:21:37 node2 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6636" x-info="http://www.rsyslog.com"] start
Jul 14 02:21:37 node2 systemd: Stopping System Logging Service...
Jul 14 02:21:37 node2 systemd: Starting System Logging Service...
Jul 14 02:21:37 node2 systemd: Started System Logging Service.
五、rsyslog + loganalyzer
loganazer 是一个PHP写的日志分析、展现程序,运行需要LAMP环境。
由ryslog负责收集日志、loganazer负责分析、展现、MySQL负责存储日志。
loganalyzer 官网: http://loganalyzer.adiscon.com/
下面我们搭建个rsyslog+loganalyzer试试水:
node1 : 192.168.80.10 LAMP、Loganalyzer、RsyslogServer、RsyslogClient
node2 : 192.168.80.11 RsyslogClient
1. 安装LAMP运行环境
[root@node1 ~]# yum -y install httpd php php-mysql mariadb mariadb-server
2. 安装Loganalyzer
#安装mysql模块,用于rsyslog驱动MySQL
[root@node1 ~]# yum -y install rsyslog-mysql
[root@node1 ~]# vim /etc/rsyslog.conf
#### MODULES #### 加载MySQL模块,必须要在MODULES段中
$ModLoad ommysql
#开启TCP/514 、UDP/514端口用于收集日志
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
#将收集到的所有日志都发往MySQL
*.* :ommysql:192.168.80.10,RsyslogDB,rsyslog,123
#创建用户、数据库
MariaDB [(none)]> CREATE DATABASE RsyslogDB;
MariaDB [(none)]> GRANT ALL ON RsyslogDB.* TO 'rsyslog'@'%' IDENTIFIED BY '123';
#查看MySQL模块中的文件,并导入sql脚本
[root@node1 ~]# rpm -ql rsyslog-mysql
/usr/lib64/rsyslog/ommysql.so
/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
#注意:这个脚本会创建数据库,根据自己情况修改此脚本,我上面已经创建过数据RsyslogDB了所以改动内容如下:
[root@node1 ~]# vim rsyslog-mysql.sql
USE RsyslogDB;
CREATE TABLE SystemEvents
(
ID int unsigned not null auto_increment primary key,
CustomerID bigint,
ReceivedAt datetime NULL,
DeviceReportedTime datetime NULL,
Facility smallint NULL,
Priority smallint NULL,
FromHost varchar(60) NULL,
Message text,
NTSeverity int NULL,
Importance int NULL,
EventSource varchar(60),
EventUser varchar(60) NULL,
EventCategory int NULL,
EventID int NULL,
EventBinaryData text NULL,
MaxAvailable int NULL,
CurrUsage int NULL,
MinUsage int NULL,
MaxUsage int NULL,
InfoUnitID int NULL ,
SysLogTag varchar(60),
EventLogType varchar(60),
GenericFileName VarChar(60),
SystemID int NULL
);
CREATE TABLE SystemEventsProperties
(
ID int unsigned not null auto_increment primary key,
SystemEventID int NULL ,
ParamName varchar(255) NULL ,
ParamValue text NULL
);
[root@node1 ~]# mysql -ursyslog -p123 -D RsyslogDB ‘loganalyzer-4.1.6’
[root@node1 html]# chown -R apache loganalyzer
[root@node1 ~]# touch /var/www/html/loganalyzer/config.php
[root@node1 html]# chmod 666 /var/www/html/loganalyzer/config.php
#重启服务
[root@node1 html]# systemctl restart mariadb httpd rsyslog
3. 客户端配置
[root@node2 ~]# vim /etc/rsyslog.conf
*.* @192.168.80.10:514
[root@node2 ~]# systemctl restart rsyslog