经排查发现本地的测试环境磁盘空间满了,造成数据库不能写入数据.

最终的罪魁祸首是日志文件.

由于我们使用的是lvs负载均衡,前端的Keepalived启动之后不能初始化ipvs协议,所以日志在不停的刷一下的内容,速度之快令人乍舌,日志文件的大小在不停的疯长,最终导致空间不足,杯具的事情就这样发生了.

错误内容

查看系统日志文件内容

tail -f /var/log/messages

你会看见很多如下的信息在不停的跳动.

May 16 08:35:22 linux-node1 Keepalived_healthcheckers: IPVS: Can't initialize ipvs: Protocol not available
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Netlink reflector reports IP 10.20.23.144 added
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Netlink reflector reports IP 10.20.23.146 added
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Registering Kernel netlink reflector
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Registering Kernel netlink command channel
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Configuration is using : 13073 Bytes
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: IPVS: Can't initialize ipvs: Protocol not available
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Netlink reflector reports IP 10.20.23.144 added
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Netlink reflector reports IP 10.20.23.146 added
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Registering Kernel netlink reflector
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Registering Kernel netlink command channel
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
May 16 08:35:22 linux-node1 Keepalived_healthcheckers: Configuration is using : 13073 Bytes

看这句"IPVS: Can't initialize ipvs: Protocol not available",已经很是明显的指出了不能初始化ipvs协议,看看解决办法吧

解决办法

首先你必须得先安装ipvsadm

yum install ipvsadm

要是你已经安装过了,这句就直接忽略掉吧,之后在执行一下命令

ipvsadm

这一个小小的命令就能够给你解决上边的问题了,现在你再看看你的message里边还有没有一直在刷新,哈哈~~~

[root@linux-node1 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.20.23.146:http wrr persistent 300
  -> web02:0                      Route   1      0          0         
  -> 10.20.23.250:0               Route   1      0          0