rsyslog特性
- 存储日志信息于MySQL,PostgreSQL,MongoDB,ElasticSearch等数据管理系统
- 通过 RELP + TCP 实现数据的可靠传输(基于此结合丰富的过滤条件可以建立一种 可靠的数据传输通道供其他应用来使用)
- 精细的输出格式控制以及对消息的强大 过滤能力
- 高精度时间戳;队列操作(内存,磁盘以及混合模式等); 支持数据的加密和压缩传输等
程序环境
主程序:rsyslogd
主配置文件:/etc/rsyslog.conf,/etc/rsyslog.d/*.conf
服务脚本(centos6):/etc/rc.d/init.d/rsyslog
Unit File(centos7):/usr/lib/systemd/system/rsyslog.service
配置文件
#rsyslog v3 config file
# if you experience problems, check
# http://www.rsyslog.com/troubleshoot for assistance
#### MODULES ####
$ModLoad imuxsock.so # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so # provides kernel logging support (previously done by rklogd)
#$ModLoad immark.so # provides --MARK-- message capability
# Provides UDP syslog reception
#$ModLoad imudp.so
#$UDPServerRun 514
# Provides TCP syslog reception
#$ModLoad imtcp.so
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
配置文件查看less /etc/rsyslog.conf
。Rsyslog的配置主要有以下模块:
- modules,模块,配置加载的模块,如:
ModLoad imudp.so
配置加载UDP传输模块 - global directives,全局配置,配置ryslog守护进程的全局属性,比如主信息队列大小(MainMessageQueueSize)
- rules,规则(选择器+动作),每个规则行由两部分组成,selector部分和action部分,这两部分由一个或多个空格或tab分隔,selector部分指定源和日志等级,action部分指定对应的操作
常用的modules
- imudp,传统方式的UDP传输,有损耗
- imtcp,基于TCP明文的传输,只在特定情况下丢失信息,并被广泛使用
- imrelp,RELP传输,不会丢失信息,但只在rsyslogd 3.15.0及以上版本中可用
规则(rules)
规则的选择器(selectors)
selector也由两部分组成,设施和优先级,由点号.分隔。多个选择器用;分隔,如:*.info;mail.none
facility:设施,从功能或程序上对日志收集进行分类
内建的日志设施有:
auth(security), authpriv:认证授权和安全相关的消息
kern:来自Linux内核的消息
mail:邮件服务系统的各种组件
mark:防火墙标记
cron:周期任务计划相关的信息
daemon:守护进程产生的信息
news:网络消息子系统
lpr:打印相关的日志信息
user:用户进程相关的信息
local0 to local7:保留,本地使用
priority:优先级,日志级别(由低到高)
debug:包含详细的开发情报的信息,通常只在调试一个程序时使用
info:情报信息,正常的系统消息,比如骚扰报告,带宽数据等,不需要处理
notice: 不是错误情况,也不需要立即处理
warning: 警告信息,不是错误,比如系统磁盘使用了85%等
err:错误,不是非常紧急,在一定时间内修复即可
crit:重要情况,如硬盘错误,备用连接丢失
alert:应该被立即改正的问题,如系统数据库被破坏,ISP连接丢失
emerg:紧急情况,需要立即通知技术人员
指定级别 | 功能 |
---|---|
* | 所有级别 |
none | 没有级别 |
priority | 此级别及高于此级别的所有级别 |
=priority | 仅此级别 |
动作 (action)
action是规则描述的一部分,位于选择器的后面,规则用于处理消息。总的来说,消息内容被写到一种日志文件上,但也可以执行其他动作,比如写到数据库表中或转发到其他主机。
# The authpriv file has restricted access.
authpriv.* /var/log/secure
写到mysql数据库中
# modules, 要将日志写到mysql中需要加载ommysql模块
$ModLoad ommysql
# rule, send to mysql
#*.* :ommysql:database-server,database-name,database-userid,database-password
*.* :ommysql:127.0.0.1,Syslog,syslogwriter,topsecret
mysql和rsyslog服务搭建
安装rsyslog-mysql
[root@promote ~]# yum install rsyslog-mysql
··· ···
Complete!
[root@promote ~]# rpm -ql rsyslog-mysql
/usr/lib64/rsyslog/ommysql.so
/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
启动MySQL服务器
[root@promote ~]# vim /etc/my.cnf.d/server.cnf
添加如下:
[mysqld]
skip_name_resolve=ON
innodb_file_per_table=ON
[root@promote ~]# systemctl start mariadb.service
[root@promote ~]# mysql -uroot -hlocalhost -pcentos < /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
MariaDB [(none)]> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| Syslog |
+--------------------+
1 rows in set (0.138 sec)
MariaDB [(none)]> use Syslog;
Database changed
MariaDB [Syslog]> SHOW TABLES;
+------------------------+
| Tables_in_Syslog |
+------------------------+
| SystemEvents |
| SystemEventsProperties |
+------------------------+
2 rows in set (0.000 sec)
MariaDB [Syslog]> DESC SystemEvents;
+--------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+------------------+------+-----+---------+----------------+
| ID | int(10) unsigned | NO | PRI | NULL | auto_increment |
| CustomerID | bigint(20) | YES | | NULL | |
| ReceivedAt | datetime | YES | | NULL | |
| DeviceReportedTime | datetime | YES | | NULL | |
| Facility | smallint(6) | YES | | NULL | |
| Priority | smallint(6) | YES | | NULL | |
| FromHost | varchar(60) | YES | | NULL | |
| Message | text | YES | | NULL | |
| NTSeverity | int(11) | YES | | NULL | |
| Importance | int(11) | YES | | NULL | |
| EventSource | varchar(60) | YES | | NULL | |
| EventUser | varchar(60) | YES | | NULL | |
| EventCategory | int(11) | YES | | NULL | |
| EventID | int(11) | YES | | NULL | |
| EventBinaryData | text | YES | | NULL | |
| MaxAvailable | int(11) | YES | | NULL | |
| CurrUsage | int(11) | YES | | NULL | |
| MinUsage | int(11) | YES | | NULL | |
| MaxUsage | int(11) | YES | | NULL | |
| InfoUnitID | int(11) | YES | | NULL | |
| SysLogTag | varchar(60) | YES | | NULL | |
| EventLogType | varchar(60) | YES | | NULL | |
| GenericFileName | varchar(60) | YES | | NULL | |
| SystemID | int(11) | YES | | NULL | |
+--------------------+------------------+------+-----+---------+----------------+
24 rows in set (0.063 sec)
MariaDB [Syslog]> DESC SystemEventsProperties;
+---------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+------------------+------+-----+---------+----------------+
| ID | int(10) unsigned | NO | PRI | NULL | auto_increment |
| SystemEventID | int(11) | YES | | NULL | |
| ParamName | varchar(255) | YES | | NULL | |
| ParamValue | text | YES | | NULL | |
+---------------+------------------+------+-----+---------+----------------+
4 rows in set (0.001 sec)
MariaDB [Syslog]> GRANT ALL ON Syslog.* TO 'rsyslog'@'192.168.0.%' IDENTIFIED BY 'rsyspass';
Query OK, 0 rows affected (0.102 sec)
MariaDB [Syslog]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.044 sec)
编辑rsyslog.conf
#### MODULES ####
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
$ModLoad ommysql //添加此行
#### RULES ####
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
#*.info;mail.none;authpriv.none;cron.none /var/log/messages
*.info;mail.none;authpriv.none;cron.none :ommysql:192.168.0.104,Syslog,rsyslog,rsyspass //添加此行
[root@promote ~]# systemctl restart rsyslog.service
搭建loganalyzer
在rsyslog服务器上搭建loganalyzer:
[root@rsyslog ~]# wget http://download.adiscon.com/loganalyzer/loganalyzer-4.1.6.tar.gz
[root@rsyslog syslog]# yum install -y httpd php php-mysql php-gd
[root@rsyslog ~]# tar xf loganalyzer-4.1.6.tar.gz
[root@rsyslog ~]# cp -r loganalyzer-4.1.6/src/* /var/www/html/syslog
[root@rsyslog ~]# cp -r loganalyzer-4.1.6/contrib/*.sh /var/www/html/syslog/
[root@rsyslog ~]# cd /var/www/html/syslog/
[root@rsyslog syslog]# chmod +x *.sh
[root@rsyslog syslog]# ./configure.sh
[root@rsyslog syslog]# chmod 666 config.php
[root@rsyslog syslog]# systemctl start httpd.service
[root@promote syslog]# systemctl stop firewalld
[root@promote syslog]# setenforce 0