Linux中日志采集以及格式定义

一、rsyslog日志

       Linux内核由很多的子系统组成，包含网络、文件访问、内存管理等，子系统需要给用户传送一些消息，这些消息内容包括消息的重要来源以及重要性等，所有这些子系统都要把消息传从到一个可以维护的公共消息区，于是产生了rsyslog。

1.日志类型

auth          #用户登陆日志（pam生日志）
authpriv      #服务认证日志（sshd认证）
kern          #内核日志
cron          #定时任务日志
lpr           #打印机日志
mail          #邮件日志
news          #新闻
user          #用户相关程序日志
local  1-7    #用户自定义日志

2.日志级别

debug              #系统调试信息
info               #常规信息
warning            #警告信息
err                #报错（级别低，阻止了某个工作不能正常工作）
crit               #报错（级别高，阻止了整个软件或整个系统不能正常工作）
alert              #需要立即修改的信息
emerg              #内核崩溃
none               #不采集任何日志信息

3.系统常用日志

/var/log/messages     #所有日志级别的常规信息（不包含邮件，服务认证，定时任务)
/var/log/maillog      #邮件认证
/var/log/secure       #服务认证日志
/var/log/cron         #定时任务日志

4.管理日志的配置文件

vim /etc/rsyslog.conf       #查看并修改采集日志文件配置，使我们能够采集日志到指定位置

二、日志采集

1.接收方测试环境调试

[root@node1 Desktop]# systemctl stop firewalld.service         ##首先关闭火墙
[root@node1 Desktop]# vim /etc/rsyslog.conf                    ##接收方打开UDP配置

 

[root@node1 Desktop]# systemctl restart rsyslog.service     ##重启日志
[root@node1 Desktop]# > /var/log/messages                   ##清空日志
[root@node1 Desktop]# cat /var/log/messages                 ##查询日志
Oct 16 09:14:00 node1 rc.local: /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:14:00 node1 rc.local: /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host
Oct 16 09:14:03 node1 rc.local: /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:14:03 node1 rc.local: /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host

2.发送方实验调试 

[root@node2 Desktop]# vim /etc/rsyslog.conf 
[root@node2 Desktop]# systemctl restart rsyslog.service 
                             ##接收方：第一步关闭火墙，打开UDP配置，重新启动
                             ##udp协议 当设定完成后 发现无法传输日志，需要设设定防火墙
                             ##关闭防火墙就  systemcrl stop firewalld.service

 

[root@node2 Desktop]# > /var/log/messages 
[root@node2 Desktop]# logger text
[root@node2 Desktop]# cat /var/log/messages 
Oct 16 09:20:29 node2 rc.local: /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:20:29 node2 rc.local: /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host
Oct 16 09:20:30 node2 root: text

3.接受方查看

[root@node1 Desktop]# > /var/log/messages 
[root@node1 Desktop]# cat /var/log/messages 
Oct 16 09:25:50 node2 rc.local: /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:25:50 node2 rc.local: /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host

4.自定义日志格式设定

[root@node1 Desktop]# vim /etc/rsyslog.conf 
[root@node1 Desktop]# systemctl restart rsyslog

 

在47行加入$template messages,"%timegenerated%  %FROMHOST—IP%  %syslogtag%  %msg%\n"

[root@node1 Desktop]# > /var/log/messages
[root@node1 Desktop]# logger test
[root@node1 Desktop]# cat /var/log/messages 
Oct 16 09:47:12  127.0.0.1  rc.local:  /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:47:12  127.0.0.1  rc.local:  /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host
Oct 16 09:47:14  127.0.0.1  root:  test
Oct 16 09:47:16  172.25.254.200  rc.local:   /etc/rc.d/rc.local: connect: No route to host
Oct 16 09:47:16  172.25.254.200  rc.local:   /etc/rc.d/rc.local: line 18: /dev/tcp/content.example.com/80: No route to host

注意：以上实验模式经常清空日志，在实际中操作切忌请勿随便清空

三、systemd-journald的日志

1..systemd-journald可移植性高，资源消耗少、结构简单、可扩展、安全性高等优点。

2.安全性之所以高是因为此程序只负责对日志进行查看而不能对日志进行保存和采集,那么关机后再开机（重启），只能查看到开机后的日志，因为系统之前的日志是保存在内存中的，所以关机后就被清空了，那么再开机时用journalctl是看不到的之前的日志信息的。

journalctl命令
格式： journalctl + 参数       #日志的查看
具体参数的含义：
 -p   err                        #查看报错日志
 -f                              #监控 （用户ctrl+c结束监控）
 -n      3                       #最新（new）的三条日志
 -o   verbose                    #查看日志详细参数
 _PID=***                        #直接查看PID为**的程序产生的日志
 --since  时间点 --until  时间点   #查看从何时到何时的日志

3.命令练习

[root@node1 Desktop]# journalctl -n 3
-- Logs begin at Tue 2018-10-16 08:51:42 EDT, end at Tue 2018-10-16 11:05:45 EDT
Oct 16 11:05:42 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/cont
Oct 16 11:05:45 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to h
Oct 16 11:05:45 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/cont

[root@node1 Desktop]# journalctl -p err
-- Logs begin at Tue 2018-10-16 08:51:42 EDT, end at Tue 2018-10-16 11:08:36 ED
Oct 16 08:51:42 localhost kernel: Detected CPU family 6 model 61
Oct 16 08:51:42 localhost kernel: Warning: Intel CPU model - this hardware has 
Oct 16 08:51:43 localhost rpcbind[196]: rpcbind terminating on signal. Restart 
Oct 16 08:51:46 node1 smartd[617]: Problem creating device name scan list
Oct 16 08:51:46 node1 smartd[617]: In the system's table of devices NO devices 
Oct 16 08:51:49 node1 rpcbind[1341]: Cannot open '/var/lib/rpcbind/rpcbind.xdr'
Oct 16 08:51:49 node1 rpcbind[1341]: Cannot open '/var/lib/rpcbind/portmap.xdr'
Oct 16 08:51:49 node1 systemd[1]: Failed to start LSB: Starts the Spacewalk Dae
Oct 16 08:52:09 node1 libvirtd[1345]: libvirt version: 1.1.1, package: 29.el7 (
Oct 16 08:52:09 node1 libvirtd[1345]: Module /usr/lib64/libvirt/connection-driv

[root@node1 Desktop]# journalctl _PID=5535
-- Logs begin at Tue 2018-10-16 08:51:42 EDT, end at Tue 2018-10-16 11:15:58 ED

[root@node1 Desktop]# journalctl --since 11:00 --until 11:01
-- Logs begin at Tue 2018-10-16 08:51:42 EDT, end at Tue 2018-10-16 11:19:31 ED
Oct 16 11:00:00 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:00 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:01 node1 systemd[1]: Starting Session 16 of user root.
Oct 16 11:00:01 node1 systemd[1]: Started Session 16 of user root.
Oct 16 11:00:01 node1 CROND[5151]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Oct 16 11:00:03 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:03 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:09 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:09 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:12 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:12 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:18 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:18 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:21 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:21 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con
Oct 16 11:00:27 node1 rc.local[1347]: /etc/rc.d/rc.local: connect: No route to 
Oct 16 11:00:27 node1 rc.local[1347]: /etc/rc.d/rc.local: line 18: /dev/tcp/con

4.让systemd-journald将日志保存到硬盘中，也就是说即使系统重启，仍然可以查看到系统重启之前的日志信息

[root@node1 Desktop]# mkdir /var/log/journal
[root@node1 Desktop]# ll -d /var/log/journal/
drwxr-xr-x. 2 root root 6 Oct 16 11:30 /var/log/journal/
[root@node1 Desktop]# chgrp systemd-journal /var/log/journal    ##更改目录的所有组
##/var/log/journal/中的日志信息只能被systemd-journal组所识别
[root@node1 Desktop]# ll -d /var/log/journal/
drwxr-xr-x. 2 root systemd-journal 6 Oct 16 11:30 /var/log/journal/
[root@node1 Desktop]# chmod g+s /var/log/journal
##使在/var/log/journal目录中出现的文件的所有组自动归属于该目录的所有组
[root@node1 Desktop]# ll -d /var/log/journal/
drwxr-sr-x. 2 root systemd-journal 6 Oct 16 11:30 /var/log/journal/
[root@node1 Desktop]# ps aux | grep systemd-journald
root       409  0.0  0.2  42992  4692 ?        Ss   08:51   0:01 /usr/lib/systemd/systemd-journald
root      6116  0.0  0.0 112644   984 pts/0    R+   11:31   0:00 grep --color=auto systemd-journald
[root@node1 Desktop]# kill -1 409  ##重新加载

[root@node1 Desktop]# date
Tue Oct 16 11:34:38 EDT 2018
[root@node1 Desktop]# reboot 
Connection to 172.25.254.100 closed by remote host.
Connection to 172.25.254.100 closed.
[kiosk@foundation69 ~]$ ssh [email protected]
[email protected]'s password: 
Last login: Tue Oct 16 08:53:50 2018 from 172.25.254.69
[root@node1 ~]# journalctl