网络设备配置Syslog日志服务器

网络设备配置日志服务器

1 环境介绍

1.1 硬件

1)     华为防火墙，型号USG5120

2)     PC机作为日志服务器，型号清华同方，硬盘1TB

1.2 软件

注意：centos 7 32bit，有些软件不能通过yum命令来安装，centos 7 已经不支持这些32位的软件了。

PC机：

1)     操作系统：Centos 7 64bit

2)     Syslog 8.24

3)     loganalyzer-4.1.6.tar

4)     Mariadb 5.5.56

5)     LAMP环境

2 日志服务器配置

2.1 安装Centos

利用之前做好的U盘PE镜像，插入PC机USB接口，开机进入PE，选择安装Centos 7 ，按照提示一步一步装好。

2.1.1 时区设置问题

[root@localhost ~]# date

Tue Mar 20 19:42:22 CST 2018

[root@localhost ~]# timedatectl

     Local time: Tue 2018-03-20 19:42:47 CST

 Universal time: Tue 2018-03-20 11:42:47 UTC

       RTC time: Tue 2018-03-20 11:42:47

      Time zone: Asia/Shanghai (CST, +0800)

     NTPenabled: no

NTP synchronized: no

 RTC inlocal TZ: no

      DSTactive: n/a

 

在安装的时候，已经选择了Asia/Shanghai (CST, +0800)，但是在使用date命令的时候，发现Local time比当前实际时间要快8个小时，在后期的日志中会有很大的问题，所以需要把时间改回来。

解决办法：

1)     先查看系统时区信息

2)     删除当前系统所处的时区

3)     从/usr/share/zoneinfo/中创建软连接以替换当前的时区信息，直接选择Universal

4)     利用timedatectl查看，OK问题解决。

[root@localhost ~]# ls/usr/share/zoneinfo/

Africa     Australia  Cuba     Etc     GMT-0      Indian       Kwajalein  Navajo   posix       ROK        UTC

America    Brazil     EET      Europe  GMT+0      Iran         Libya      NZ       posixrules  Singapore  WET

Antarctica Canada     Egypt    GB      Greenwich  iso3166.tab  MET       NZ-CHAT   PRC         Turkey     W-SU

Arctic     CET        Eire     GB-Eire Hongkong   Israel       Mexico     Pacific  PST8PDT     UCT        zone.tab

Asia       Chile      EST      GMT     HST        Jamaica      MST       Poland    right       Universal  Zulu

Atlantic   CST6CDT    EST5EDT  GMT0    Iceland    Japan        MST7MDT    Portugal ROC         US

[root@localhost ~]# rm /etc/localtime

rm: remove symbolic link ‘/etc/localtime’?

[root@localhost ~]# rm/etc/localtime

rm: remove symbolic link ‘/etc/localtime’? y

[root@localhost ~]# ln-s /usr/share/zoneinfo/Universal /etc/localtime

[root@localhost ~]#timedatectl

     Local time: Tue 2018-03-20 11:48:29 UTC

 Universal time: Tue 2018-03-20 11:48:29 UTC

       RTC time: Tue 2018-03-20 11:48:28

      Time zone: Universal (UTC, +0000)

     NTPenabled: yes

NTP synchronized: no

 RTC inlocal TZ: no

      DSTactive: n/a

[root@localhost ~]#date

Tue Mar 20 11:48:36 UTC 2018

2.1.2 时间与网络时间不一致

1.  安装ntpdate工具

# yum -y install ntp ntpdate

2.  设置系统时间与网络时间同步

# ntpdate cn.pool.ntp.org

2.2 配置网络

进入/etc/sysconfig/network-scripts/，由于我这里没有eth0，默认的是ifcfg-enp4s0，所以vi ifcfg-enp4s0，配置如下：

TYPE=Ethernet

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=yes

IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=eth0

UUID=8a67b4da-3bb2-40df-9006-9c3fd9ec7b95

DEVICE=enp4s0

ONBOOT=yes

HWADDR=34:97:F6:9A:5E:94

IPADDR=192.168.1.15

PREFIX=24

GATEWAY=192.168.1.10

DNS1=114.114.114.114

IPV6_PEERDNS=yes

IPV6_PEERROUTES=yes

IPV6_PRIVACY=no

2.2.1 能ping通IP，但不能访问域名

[root@localhost ~]# ping 192.168.1.10

PING 192.168.1.10 (192.168.1.10) 56(84) bytesof data.

64 bytes from 192.168.1.10: icmp_seq=1 ttl=64time=0.443 ms

 

[root@localhost ~]# ping www.baidu.com

ping: www.baidu.com: Name orservice not known

 

解决办法：

将/etc/resovl.conf文件设置成：

search localdomain

nameserver 114.114.114.114

 

2.3 在rsyslog服务器上部署 LAMP环境

2.3.1 安装httpd

[root@rsyslog ~]# yum install httpd -y

[root@rsyslog ~]# systemctl start httpd    #启动服务

[root@rsyslog ~]# systemctl enable httpd  #设置开启启动

2.3.1.1 关闭防火墙

如果系统中开着防火墙，那么外部访问192.168.1.15/phpmyadmin就无法访问.

[root@mycentos ~]# systemctl stop firewalld.service #停止防火墙服务

[root@localhost ~]# systemctldisable firewalld.service #禁用防火墙开机启动服务

Removed symlink/etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

2.3.1.2 配置UTF8字符

[root@localhost loganalyzer]# cat /etc/httpd/conf/httpd.conf

#

# Specify a default charset for all contentserved; this enables

# interpretation of all content as UTF-8 bydefault.  To use the

# default browser choice (ISO-8859-1), or toallow the META tags

# in HTML content to override this choice,comment out this

# directive:

#

AddDefaultCharset UTF-8

2.3.2 安装mysql

由于centos 7 默认源中已经没有mysql数据库了，所以只能安装mariadb

[root@localhost conf.d]# yum -y installmariadb-server mariadb

[root@rsyslog ~]# yuminstall mysql mysql-server –y

[root@rsyslog ~]# systemctl start mariadb

[root@rsyslog ~]# systemctl enable mariadb

Created symlink from/etc/systemd/system/multi-user.target.wants/mariadb.service to/usr/lib/systemd/system/mariadb.service.

[root@rsyslog ~]# mysqladmin -u root password '2991421'

注意：在Centos7上默认安装mysql会变成mariadb，mysql的开源版

安装完了之后启动不了

[root@localhost html]#systemctl start mariadb

Failed to startmariadb.service: Unit not found

yum install -y mariadb*

2.3.3 设置UFT8字符

1)     文件/etc/my.cnf

vi /etc/my.cnf

 

在[mysqld]标签下添加

init_connect='SET collation_connection =utf8_unicode_ci'

init_connect='SET NAMES utf8'

character-set-server=utf8

collation-server=utf8_unicode_ci

skip-character-set-client-handshake

 

2)     文件/etc/my.cnf.d/client.cnf

vi /etc/my.cnf.d/client.cnf

在[client]中添加

default-character-set=utf8

 

3)     文件/etc/my.cnf.d/mysql-clients.cnf

vi/etc/my.cnf.d/mysql-clients.cnf

在[mysql]中添加

default-character-set=utf8

4)     全部配置完成，重启mariadb

systemctl restart mariadb

之后进入MariaDB查看字符集

mysql> show variables like"%character%";show variables like "%collation%";

显示为

 

+--------------------------+----------------------------+

| Variable_name           | Value                     |

+--------------------------+----------------------------+

| character_set_client    |utf8                     |

| character_set_connection | utf8                     |

| character_set_database  | utf8                     |

| character_set_filesystem | binary                   |

| character_set_results    |utf8                     |

| character_set_server    |utf8                     |

| character_set_system    |utf8                     |

| character_sets_dir      |/usr/share/mysql/charsets/ |

+--------------------------+----------------------------+

8 rows in set (0.00 sec)

+----------------------+-----------------+

| Variable_name        |Value          |

+----------------------+-----------------+

| collation_connection | utf8_unicode_ci |

| collation_database  | utf8_unicode_ci |

| collation_server    |utf8_unicode_ci |

+----------------------+-----------------+

3 rows in set (0.00 sec)

字符集配置完成。

 

2.3.4 移动mysql的数据库文件存放位置

由于日志量很大，默认安装的目录下硬盘空间有限，需要将此目录移动到单独划出来的一块大磁盘空间上。

[root@localhostmysql]# df -h

Filesystem               Size  Used Avail Use% Mounted on

/dev/mapper/centos-root   50G 2.6G   48G   6% /

devtmpfs                 1.9G     0 1.9G   0% /dev

tmpfs                    1.9G     0 1.9G   0% /dev/shm

tmpfs                    1.9G  8.8M 1.9G   1% /run

tmpfs                    1.9G     0 1.9G   0% /sys/fs/cgroup

/dev/sda1               1014M  139M 876M  14% /boot

/dev/mapper/centos-home  877G  62M  877G   1% /home

tmpfs                    383M     0 383M   0% /run/user/0

 

默认的存放目录为/var/lib/mysql，更改后为/home/data/mysql

注意目录的属主和权限。 chown -R mysql:mysql /home/data/

 

1)     创建目录

a)      cd /home

b)     mkdir data

2)     停止数据库服务

a)      [root@localhost mysql]# systemctlstop mariadb  

3)     把/var/lib/mysql整个目录保持权限复制到/home/data

a)      cp -arp /var/lib/mysql　/home/data/

4)     修改配置文件

a)      [root@localhost ~]# vi/etc/my.cnf.d/server.cnf

b)     [mysqld]

c)      datadir=/home/data/mysql

d)     socket=/var/lib/mysql/mysql.sock

e)      character_set_server=utf8

f)      slow_query_log=on

g)     slow_query_log_file=/home/data/mysql/slow_query_log.log

h)     long_query_time=2

5)     配置慢查询

a)      touch /home/data/mysql /slow_query_log.log

b)     chown mysql:mysql /home/data/mysql /slow_query_log.log

6)     关闭selinux

a)      setenforce 0

7)     重启mysql

a)      systemctl restart mariadb

b)     mysql –u root –p

c)      新建一个数据库，查看是否在新目录下已经生成

2.3.5 安装php

[root@rsyslog ~]# yum install php php-mysqlphp-gd -y

[root@rsyslog ~]# vi /var/www/html/test.php

phpinfo();

?>

 

打开客户端浏览器，进行访问测试http://192.168.1.15/test.php

若是不能访问，那么重启Apache服务.

systemctl stop httpd

systemctl start httpd

 

注意：Apache默认的网站服务地址为/var/www/html/下面，如果不存在可以新建。若是Apache中，php网页不能正确运行，请确保httpd中开启了对php的支持，

命令：httpd–M

2.3.6 php 5.4升级到5.6

Centos 7 使用yum默认安装的是php 5.4版本的，但是最新的phpmyadmin不支持这个版本的php，只支持php5.5及以上的版本.

 

更新之前查看当前PHP版本避免重复更新

# php -v　　

 

检查当前PHP的安装包

# yum list installed | grep php

 

完全移除当前PHP安装包以免起冲突

# yum remove php*

 

默认的yum源无法升级PHP，需要添加第三方yum源，我们选择webtatic库

CentOs 7.X

rpm -Uvhhttps://mirror.webtatic.com/yum/el7/epel-release.rpm

rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm

 

安装php5.6

yum install -y php56w.x86_64 php56w-cli.x86_64php56w-common.x86_64 php56w-gd.x86_64 php56w-ldap.x86_64 php56w-mbstring.x86_64php56w-mcrypt.x86_64 php56w-mysql.x86_64 php56w-pdo.x86_64 php56w-fpm

 

安装php7.1

yum install -y php71w-fpm php71w-opcachephp71w-cli php71w-gd php71w-imap php71w-mysqlnd php71w-mbstring php71w-mcryptphp71w-pdo php71w-pecl-apcu php71w-pecl-mongodb php71w-pecl-redis php71w-pgsqlphp71w-xml php71w-xmlrpc php71w-devel mod_php71w

 

查看当前php版本

php –v

2.3.7 安装phpmyadmin

2.3.7.1 安装

[root@localhost ~]# yum install phpmyadmin

有时候会安装不成功，提示没有可用软件包，则需要安装Remi源 

# yum install epel-release

# rpm -ivh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm

[root@localhost ~]# yum install phpmyadmin

Yum默认安装在/usr/share/phpMyAdmin目录下

2.3.7.2 配置

[root@localhost conf.d]# vi /etc/httpd/conf.d/phpMyAdmin.conf

只需要修改红色字体部分就可以了

# phpMyAdmin - Web based MySQL browser writtenin php

#

# Allows only localhost by default

#

# But allowing phpMyAdmin to anyone other thanlocalhost should be considered

# dangerous unless properly secured by SSL

 

Alias /phpMyAdmin /usr/share/phpMyAdmin

Alias /phpmyadmin /usr/share/phpMyAdmin

 

  AddDefaultCharset UTF-8

 

  

     #Apache 2.4

    

       #Require ip 127.0.0.1

       #Require ip ::1

        Require all granted

    

  

  

     #Apache 2.2

    Order Deny,Allow

     Denyfrom All

     #Allow from 127.0.0.1

     #Allow from ::1

     Allow from All granted

  

 

  

     #Apache 2.4

    

      Require ip 127.0.0.1

      Require ip ::1

    

  

  

     #Apache 2.2

    Order Deny,Allow

     Denyfrom All

    Allow from 127.0.0.1

    Allow from ::1

  

 

# These directories do not require access overHTTP - taken from the original

# phpMyAdmin upstream tarball

#

    OrderDeny,Allow

    Denyfrom All

    Allowfrom None

 

    OrderDeny,Allow

    Denyfrom All

    Allowfrom None

 

    OrderDeny,Allow

    Denyfrom All

    Allowfrom None

 

# This configuration prevents mod_security atphpMyAdmin directories from

# filtering SQL etc.  This may break your mod_securityimplementation.

#

#

#    

#       SecRuleInheritance Off

#   

#

2.3.7.3 测试

重启httpd服务

[root@localhost conf.d]# systemctl restarthttpd

在浏览器中访问http://192.168.1.15/phpmyadmin/index.php

2.4 安装或者升级Rsyslog，支持mysql数据库

[root@localhost ~]# yum install rsyslog-*--skip-broken –y

[root@localhost ~]# systemctl enable rsyslog

2.5 在mariadb中创建Rsyslog数据库

rsyslog建库脚本在/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql，直接将这个脚本导入mariadb数据中就好了。

2.5.1 导入脚本

[[email protected]]# mysql -uroot -p

2.5.2 验证是否导入成功

去数据库中查看数据库是否存在：

MariaDB [(none)]>show databases;   

+--------------------+

| Database           |

+--------------------+

| information_schema|

| Syslog             |

| mysql              |

| performance_schema|

| test               |

+--------------------+

5 rows in set (0.00sec)

 

查看Syslog中的tables：

MariaDB [Syslog]>show tables;

+------------------------+

|Tables_in_Syslog       |

+------------------------+

| SystemEvents           |

|SystemEventsProperties |

+------------------------+

2 rows in set (0.00sec)

2.5.3 创建用户并赋权限

MariaDB [Syslog]> GRANT ALL ON Syslog.* TOsyslog@localhost IDENTIFIED BY 'syslog';

给Syslog数据库添加用户名为syslog，密码为syslog的用户。红色部分为密码。

MariaDB [Syslog]> flush privileges;

MariaDB [Syslog]> exit

2.5.4 验证用户是否能够登陆

[root@localhost rsyslog-8.24.0]# mysql -usyslog –p

Enter password:

MariaDB [(none)]> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| Syslog             |

| test               |

+--------------------+

3 rows in set (0.00 sec)

2.6 修改rsyslog的配置

[root@localhost rsyslog-8.24.0]# vi/etc/rsyslog.conf

 

# rsyslog configuration file

 

# For more information see/usr/share/doc/rsyslog-*/rsyslog_conf.html

# If you experience problems, seehttp://www.rsyslog.com/doc/troubleshoot.html

 

#### MODULES ####

 

# The imjournal module bellow is now usedas a message source instead of imuxsock.

$ModLoad imuxsock # provides support for local system logging (e.g. via loggercommand)

$ModLoad imjournal # provides access to the systemd journal

$ModLoad imklog # reads kernel messages (the same are read from journald)

$ModLoad immark  # provides --MARK-- messagecapability

 

# Provides UDP syslog reception

$ModLoad imudp

$UDPServerRun 514

 

# Provides TCP syslog reception

$ModLoad imtcp

$InputTCPServerRun 514

 

$ModLoad ommysql

*.*:ommysql:localhost,Syslog,syslog,syslog

 

#### GLOBAL DIRECTIVES ####

 

# Where to place auxiliary files

$WorkDirectory /var/lib/rsyslog

 

# Use default timestamp format

$ActionFileDefaultTemplateRSYSLOG_TraditionalFileFormat

 

# File syncing capability is disabled bydefault. This feature is usually not required,

# not useful and an extreme performance hit

#$ActionFileEnableSync on

 

# Include all config files in/etc/rsyslog.d/

$IncludeConfig /etc/rsyslog.d/*.conf

 

# Turn off message reception via local logsocket;

# local messages are retrieved throughimjournal now.

$OmitLocalLogging on

 

# File to store the position in the journal

$IMJournalStateFile imjournal.state

 

 

#### RULES ####

 

# Log all kernel messages to the console.

# Logging much else clutters up the screen.

#kern.*                                                 /dev/console

 

# Log anything (except mail) of level infoor higher.

# Don't log private authenticationmessages!

#*.info;mail.none;authpriv.none;cron.none                /var/log/messages

 

# The authpriv file has restricted access.

#authpriv.*                                              /var/log/secure

 

# Log all the mail messages in one place.

#mail.*                                                  -/var/log/maillog

 

 

# Log cron stuff

#cron.*                                                  /var/log/cron

 

 

重启rsyslog服务

[root@localhost loganalyzer]# systemctl stoprsyslog

[root@localhost loganalyzer]# systemctl startrsyslog

2.7 Loganalyzer

2.7.1 下载

最新的下载地址: http://loganalyzer.adiscon.com/downloads/

找到最新版版本，然后点击下载，在360下载器中获得下载地址，复制到SecureCRT中

[root@localhost loganalyzer]# wget -chttp://download.adiscon.com/loganalyzer/loganalyzer-4.1.6.tar.gz

[root@rsyslog ~]# tar zxvf loganalyzer-4.1.6.tar.gz

复制文件到

[root@rsyslog ~]# cp -r loganalyzer-4.1.6/src//var/www/html/loganalyzer

[root@rsyslog ~]# cp -r loganalyzer-4.1.6/contrib/*/var/www/html/loganalyzer/

进入到loganalyzer 程序中，将以下脚本赋予执行权限

[root@rsyslog ~]# cd /var/www/html/loganalyzer/

[root@rsyslog loganalyzer]# chmod +xconfigure.sh secure.sh

运行configure.sh，将创建一个空白的config.php配置文件

[root@rsyslog loganalyzer]# ./configure.sh

2.7.2 在浏览器安装向导中安装LogAnalyzer

打开浏览器访问：http://192.168.1.15/loganalyzer/

选择here进行安装

 

命令：setenforce 0，关闭selinux。

否则会提示At least one file or directory (or more) is notwriteable, please check the file permissions (chmod 666)!,也就是config.php文件没有写的权限。

在后台数据数据库中可以看到新建出来的用户表：

loganalyzer

MariaDB [Syslog]>show tables;

+------------------------+

| Tables_in_Syslog       |

+------------------------+

| SystemEvents           |

|SystemEventsProperties |

| logcon_charts          |

| logcon_config          |

|logcon_dbmappings      |

| logcon_fields          |

|logcon_groupmembers    |

| logcon_groups          |

| logcon_savedreports    |

|logcon_searches        |

| logcon_sources         |

| logcon_users           |

| logcon_views           |

+------------------------+

13 rows in set (0.00sec)

2.7.3 UTF8设置

 

 

附录：

查看mysql表占用的空间大小：

 select concat(round(sum(data_length/1024/1024),2),'MB')as data_length_MB,concat(round(sum(index_length/1024/1024),2),'MB') asindex_length_MB from tables where table_schema='dbname' and table_name ='tablename';  

 

selectconcat(round(sum(data_length/1024/1024),2),'MB') as data_length_MB,concat(round(sum(index_length/1024/1024),2),'MB')as index_length_MB from tables where table_schema='Syslog'and table_name = 'SystemEvents';  

 

在log中添加IP字段

 

使用rsyslog+loganalzey收集日志显示客户端ip

1、数据库修改

LogAnalyzer 默认表字段只有一个 FromHost，我们在添加一个 FromIP，用于记录源IP地址。

mysql> use Syslog;

mysql> alter table SystemEvents add FromIP varchar(60) default null after FromHost; 

2、修改rsyslog.conf

rsyslog 默认情况下插入语句没有 FromIP字段，我们修改插入SQL 语句添加 FromIP字段即可。

$template insertpl,"insert into SystemEvents (Message, Facility, FromHost, FromIP, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', '%fromhost-ip%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL

$ModLoad ommysql 

 *.*       :ommysql:localhost,Syslog,rsyslog,syslog;insertpl 

#应用上面SQL语句，template在ModLoad上面

3、LogAnalyzer添加源IP地址

3.1、登录管理后台

使用管理员账号登录

进入Admin Center

3.2、添加Fields

3.3、添加views

3.4添加DBMappings,用于建立字段对应关系

注： DBMappings注意大小写，对应后面全部小写，不能有错，对应如下。

uID => id, Date => devicereportedtime, Host => fromhost, Messagetype => infounitid, Message => message, Facility => facility, Severity => priority, Syslogtag => syslogtag, ProcessID => processid, Event ID => eventid, Eventlog Type => eventlogtype, Event Source => eventsource, Event Category => eventcategory, Event User => eventuser, SystemID => systemid, Checksum => checksum

3.5、修改数据源配置

修改默认 Table type=>> MonitorWare, 修改为 NewSyslog 也就是上面新添加的NewSyslog。

修改日志选择 SelectView => NewSyslog。 

4、效果图