Mysql+Keepalived双主热备高可用操作记录

我们通常说的双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务。当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,而且切换的时间非常短。MySQL双主复制,即互为Master-Slave(只有一个Master提供写操作),可以实现数据库服务器的热备,但是一个Master宕机后不能实现动态切换。使用Keepalived,可以通过虚拟IP,实现双主对外的统一接口以及自动检查、失败切换机制,从而实现MySQL数据库的高可用方案。之前梳理了Mysql主从/主主同步,下面说下Mysql+keeoalived双主热备高可用方案的实施。

1

2

3

4

5

6

Keepalived看名字就知道,保持存活,在网络里面就是保持在线了,也就是所谓的高可用或热备,用来防止单点故障(单点故障是指一旦某一点出现故障就会导

整个系统架构的不可用)的发生,那说到keepalived不得不说的一个协议不是VRRP协议,可以说这个协议就是keepalived实现的基础。

1)Keepalived的工作原理是VRRP(Virtual Router Redundancy Protocol)虚拟路由冗余协议。在VRRP中有两组重要的概念:VRRP路由器和虚拟路由器,主控路由器和备份路由器。

2)VRRP路由器是指运行VRRP的路由器,是物理实体,虚拟路由器是指VRRP协议创建的,是逻辑概念。一组VRRP路由器协同工作,共同构成一台虚拟路由器。

Vrrp中存在着一种选举机制,用以选出提供服务的路由即主控路由,其他的则成了备份路由。当主控路由失效后,备份路由中会重新选举出一个主控路由,来继

续工作,来保障不间断服务。

过多内容在这里就不做详细介绍了,下面详细记录下Mysql+Keepalived双主热备的高可用方案的操作记录

1)先实施Master->Slave的主主同步。主主是数据双向同步,主从是数据单向同步。一般情况下,主库宕机后,需要手动将连接切换到从库上。(但是用keepalived就可以自动切换)
2)再结合Keepalived的使用,通过VIP实现Mysql双主对外连接的统一接口。即客户端通过Vip连接数据库;当其中一台宕机后,VIP会漂移到另一台上,这个过程对于客户端的数据连接来说几乎无感觉,从而实现高可用。

Mysql+Keepalived双主热备高可用操作记录_第1张图片

1

2

3

4

5

6

7

8

9

10

11

12

13

14

环境描述:

mysql的安装可以参考:http://www.cnblogs.com/kevingrace/p/6109679.html

Centos6.8版本

Master1:182.148.15.238        安装mysql和keepalived

Master2: 182.148.15.237        安装mysql和keepalived

VIP:182.148.15.236

    

要实现主主同步,可以先实现主从同步,即master1->master2的主从同步,然后master2->master1的主从同步.

这样,双方就完成了主主同步。

 

注意下面几点:

1)要保证同步服务期间之间的网络联通。即能相互ping通,能使用对方授权信息连接到对方数据库(防火墙开放3306端口)。

2)关闭selinux。

3)同步前,双方数据库中需要同步的数据要保持一致。这样,同步环境实现后,再次更新的数据就会如期同步了。

可能出现的问题

1

2

3

4

5

6

7

报错:

Last_IO_Error: Fatal error: The slave I/O thread stops because master and slave have equal MySQL server ids; these ids must be different for replication to work (or the --replicate-same-server-id option must be used on slave but this does not always make sense; please check the manual before using it).

  

解决办法:

删除mysql数据目录下的auto.cnf文件,重启mysql服务即可!

 

另:Keepalived必须使用root账号启动!!

一、Mysql主主同步环境部署

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

---------------master1服务器操作记录---------------

在my.cnf文件的[mysqld]配置区域添加下面内容:

[root@master1 ~]# vim /usr/local/mysql/my.cnf

server-id = 1         

log-bin = mysql-bin     

sync_binlog = 1

binlog_checksum = none

binlog_format = mixed

auto-increment-increment = 2     

auto-increment-offset = 1    

slave-skip-errors = all      

  

[root@master1 ~]# /etc/init.d/mysql restart

Shutting down MySQL. SUCCESS!

Starting MySQL.. SUCCESS!

  

数据同步授权(iptables防火墙开启3306端口)这样I/O线程就可以以这个用户的身份连接到主服务器,并且读取它的二进制日志。

mysql> grant replication slave,replication client on *.* to wang@'182.148.15.%' identified by "wang@123";

Query OK, 0 rows affected (0.00 sec)

  

mysql> flush privileges;

Query OK, 0 rows affected (0.00 sec)

  

最好将库锁住,仅仅允许读,以保证数据一致性;待主主同步环境部署后再解锁;

锁住后,就不能往表里写数据,但是重启mysql服务后就会自动解锁!

mysql> flush tables with read lock;  //注意该参数设置后,如果自己同步对方数据,同步前一定要记得先解锁!

Query OK, 0 rows affected (0.00 sec)

  

查看下log bin日志和pos值位置

mysql> show master status;

+------------------+----------+--------------+--------------------------+-------------------+

| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB         | Executed_Gtid_Set |

+------------------+----------+--------------+--------------------------+-------------------+

| mysql-bin.000004 |      430 |              | mysql,information_schema |                   |

+------------------+----------+--------------+--------------------------+-------------------+

1 row in set (0.00 sec)

  

---------------master2服务器操作记录---------------

在my.cnf文件的[mysqld]配置区域添加下面内容:

[root@master2 ~]# vim /usr/local/mysql/my.cnf

server-id = 2        

log-bin = mysql-bin    

sync_binlog = 1

binlog_checksum = none

binlog_format = mixed

auto-increment-increment = 2     

auto-increment-offset = 2    

slave-skip-errors = all

  

[root@master2 ~]# /etc/init.d/mysql restart

Shutting down MySQL.. SUCCESS!

Starting MySQL.. SUCCESS!

  

mysql> grant replication slave,replication client on *.* to wang@'182.148.15.%' identified by "wang@123";

Query OK, 0 rows affected (0.00 sec)

  

mysql> flush privileges;

Query OK, 0 rows affected (0.00 sec)

  

mysql> flush tables with read lock;

Query OK, 0 rows affected (0.00 sec)

  

mysql> show master status;

+------------------+----------+--------------+--------------------------+-------------------+

| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB         | Executed_Gtid_Set |

+------------------+----------+--------------+--------------------------+-------------------+

| mysql-bin.000003 |      430 |              | mysql,information_schema |                   |

+------------------+----------+--------------+--------------------------+-------------------+

1 row in set (0.00 sec)

  

---------------master1服务器做同步操作---------------

mysql> unlock tables;     //先解锁,将对方数据同步到自己的数据库中

mysql> slave stop;

mysql> change  master to master_host='182.148.15.237',master_user='wang',master_password='wang@123',master_log_file='mysql-bin.000003',master_log_pos=430;         

Query OK, 0 rows affected, 2 warnings (0.01 sec)

  

mysql> start slave;

Query OK, 0 rows affected (0.01 sec)

  

查看同步状态,如下出现两个“Yes”,表明同步成功!

mysql> show slave status \G;

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 182.148.15.237

                  Master_User: wang

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000003

          Read_Master_Log_Pos: 430

               Relay_Log_File: mysql-relay-bin.000002

                Relay_Log_Pos: 279

        Relay_Master_Log_File: mysql-bin.000003

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

        .........................

        Seconds_Behind_Master: 0

        .........................

 

这样,master1就和master2实现了主从同步,即master1同步master2的数据。

  

---------------master2服务器做同步操作---------------

mysql> unlock tables;     //先解锁,将对方数据同步到自己的数据库中

mysql> slave stop;

mysql> change  master to master_host='182.148.15.238',master_user='wang',master_password='wang@123',master_log_file='mysql-bin.000004',master_log_pos=430;  

Query OK, 0 rows affected, 2 warnings (0.06 sec)

  

mysql> start slave;

Query OK, 0 rows affected (0.01 sec)

  

mysql> show slave status \G;

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 182.148.15.238

                  Master_User: wang

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000004

          Read_Master_Log_Pos: 430

               Relay_Log_File: mysql-relay-bin.000002

                Relay_Log_Pos: 279

        Relay_Master_Log_File: mysql-bin.000004

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

        ........................

        Seconds_Behind_Master: 0

        ........................

  

这样,master2就和master1实现了主从同步,即master2也同步master1的数据。

  

以上表明双方已经实现了mysql主主同步。

当运行一段时间后,要是发现同步有问题,比如只能单向同步,双向同步失效。可以重新执行下上面的change master同步操作,只不过这样同步后,只能同步在此之后的更新数据。下面开始进行数据验证:

  

-----------------主主同步效果验证---------------------

1)在master1数据库上写入新数据

mysql> unlock tables;

Query OK, 0 rows affected (0.00 sec)

  

mysql> create database huanqiu;

Query OK, 1 row affected (0.01 sec)

  

mysql> use huanqiu;

Database changed

  

mysql> create table if not exists haha (

    -> id int(10) PRIMARY KEY AUTO_INCREMENT,

    -> name varchar(50) NOT NULL);

Query OK, 0 rows affected (0.04 sec)

  

mysql> insert into haha values(1,"王士博");

Query OK, 1 row affected (0.00 sec)

  

mysql> insert into haha values(2,"郭慧慧");

Query OK, 1 row affected (0.00 sec)

  

mysql> select * from haha;

+----+-----------+

id | name      |

+----+-----------+

|  1 | 王士博    |

|  2 | 郭慧慧    |

+----+-----------+

2 rows in set (0.00 sec)

  

然后在master2数据库上查看,发现数据已经同步过来了!

mysql> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| huanqiu            |

| mysql              |

| performance_schema |

test               |

+--------------------+

5 rows in set (0.00 sec)

  

mysql> use huanqiu;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

  

Database changed

mysql> show tables;

+-------------------+

| Tables_in_huanqiu |

+-------------------+

| haha              |

+-------------------+

1 row in set (0.00 sec)

  

mysql> select * from haha;

+----+-----------+

id | name      |

+----+-----------+

|  1 | 王士博    |

|  2 | 郭慧慧    |

+----+-----------+

2 rows in set (0.00 sec)

  

2)在master2数据库上写入新数据

mysql> create database hehe;

Query OK, 1 row affected (0.00 sec)

  

mysql> insert into huanqiu.haha values(3,"周正"),(4,"李敏");

Query OK, 2 rows affected (0.00 sec)

Records: 2  Duplicates: 0  Warnings: 0

  

然后在master1数据库上查看,发现数据也已经同步过来了!

mysql> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| hehe               |

| huanqiu            |

| mysql              |

| performance_schema |

test               |

+--------------------+

6 rows in set (0.00 sec)

  

mysql> select * from huanqiu.haha;

+----+-----------+

id | name      |

+----+-----------+

|  1 | 王士博    |

|  2 | 郭慧慧    |

|  3 | 周正      |

|  4 | 李敏      |

+----+-----------+

4 rows in set (0.00 sec)

  

至此,Mysql主主同步环境已经实现。

二、配置Mysql+Keepalived故障转移的高可用环境

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

1)安装keepalived并将其配置成系统服务。master1和master2两台机器上同样进行如下操作:

[root@master1 ~]# yum install -y openssl-devel

[root@master1 ~]# cd /usr/local/src/

[root@master1 src]# wget http://www.keepalived.org/software/keepalived-1.3.5.tar.gz

[root@master1 src]# tar -zvxf keepalived-1.3.5.tar.gz

[root@master1 src]# cd keepalived-1.3.5

[root@master1 keepalived-1.3.5]# ./configure --prefix=/usr/local/keepalived

[root@master1 keepalived-1.3.5]# make && make install

     

[root@master1 keepalived-1.3.5]# cp /usr/local/src/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/rc.d/init.d/

[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

[root@master1 keepalived-1.3.5]# mkdir /etc/keepalived/

[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

[root@master1 keepalived-1.3.5]# echo "/etc/init.d/keepalived start" >> /etc/rc.local

     

2)master1机器上的keepalived.conf配置。(下面配置中没有使用lvs的负载均衡功能,所以不需要配置虚拟服务器virtual server)

[root@master1 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

[root@master1 ~]# vim /etc/keepalived/keepalived.conf       #清空默认内容,直接采用下面配置:

! Configuration File for keepalived

       

global_defs {

notification_email {

[email protected]

[email protected]

}

       

notification_email_from [email protected]

smtp_server 127.0.0.1 

smtp_connect_timeout 30

router_id MASTER-HA

}

       

vrrp_script chk_mysql_port {     #检测mysql服务是否在运行。有很多方式,比如进程,用脚本检测等等

    script "/opt/chk_mysql.sh"   #这里通过脚本监测

    interval 2                   #脚本执行间隔,每2s检测一次

    weight -5                    #脚本结果导致的优先级变更,检测失败(脚本返回非0)则优先级 -5

    fall 2                    #检测连续2次失败才算确定是真失败。会用weight减少优先级(1-255之间)

    rise 1                    #检测1次成功就算成功。但不修改优先级

}

       

vrrp_instance VI_1 {

    state MASTER    

    interface eth0      #指定虚拟ip的网卡接口

    mcast_src_ip 182.148.15.238

    virtual_router_id 51    #路由器标识,MASTER和BACKUP必须是一致的

    priority 101            #定义优先级,数字越大,优先级越高,在同一个vrrp_instance下,MASTER的优先级必须大于BACKUP的优先级。这样MASTER故障恢复后,就可以将VIP资源再次抢回来 

    advert_int 1         

    authentication {   

        auth_type PASS 

        auth_pass 1111     

    }

    virtual_ipaddress {    

        182.148.15.236

    }

      

track_script {               

   chk_mysql_port             

}

}

     

编写切换脚本。KeepAlived做心跳检测,如果Master的MySQL服务挂了(3306端口挂了),那么它就会选择自杀。Slave的KeepAlived通过心跳检测发现这个情况,就会将VIP的请求接管

[root@master1 ~]# vim /opt/chk_mysql.sh

#!/bin/bash

counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l)

if "${counter}" -eq 0 ]; then

    /etc/init.d/keepalived stop

fi

 

[root@master1 ~]# chmod 755 /opt/chk_mysql.sh

     

启动keepalived服务

[root@master1 ~]# /etc/init.d/keepalived start

正在启动 keepalived:                                      [确定]

     

     

4)master2机器上的keepalived配置。master2机器上的keepalived.conf文件只修改priority为90、nopreempt不设置、real_server设置本地IP。

[root@master2 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

[root@master2 ~]# >/etc/keepalived/keepalived.conf

[root@master2 ~]# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

       

global_defs {

notification_email {

[email protected]

[email protected]

}

       

notification_email_from [email protected]

smtp_server 127.0.0.1 

smtp_connect_timeout 30

router_id MASTER-HA

}

       

vrrp_script chk_mysql_port {

    script "/opt/chk_mysql.sh"

    interval 2            

    weight -5                 

    fall 2                 

    rise 1               

}

       

vrrp_instance VI_1 {

    state BACKUP

    interface eth0    

    mcast_src_ip 182.148.15.237

    virtual_router_id 51    

    priority 99          

    advert_int 1         

    authentication {   

        auth_type PASS 

        auth_pass 1111     

    }

    virtual_ipaddress {    

        182.148.15.236

    }

      

track_script {               

   chk_mysql_port             

}

}

     

     

[root@master2 ~]# cat /opt/chk_mysql.sh

#!/bin/bash

counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l)

if "${counter}" -eq 0 ]; then

    /etc/init.d/keepalived stop

fi

 

[root@master2 ~]# chmod 755 /opt/chk_mysql.sh

     

[root@master2 ~]# /etc/init.d/keepalived start

正在启动 keepalived:                                      [确定]

     

     

5)master1和master2两台服务器都要授权允许root用户远程登录,用于在客户端登陆测试!

mysql> grant all on *.* to root@'%' identified by "1234567";

Query OK, 0 rows affected (0.00 sec)

     

mysql> flush privileges;

Query OK, 0 rows affected (0.01 sec)

   

6)在master1和master2两台机器上设置iptables防火墙规则,如下:

[root@master1 ~]# cat /etc/sysconfig/iptables

........

-A INPUT -s 182.148.15.0/24 -d 224.0.0.18 -j ACCEPT       #允许组播地址通信

-A INPUT -s 182.148.15.0/24 -p vrrp -j ACCEPT             #允许VRRP(虚拟路由器冗余协)通信

-A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT    #开放mysql的3306端口

   

[root@master1 ~]# /etc/init.d/iptables restart

三、Mysql+keepalived故障转移的高可用测试

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

1)通过Mysql客户端通过VIP连接,看是否连接成功。

比如,在远程一台测试机上连接,通过vip地址可以正常连接(下面的连接权限要是在服务端提前授权的)

[root@dev-new-test ~]# mysql -h182.148.15.236 -uroot -p123456

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 60

Server version: 5.6.35-log Source distribution

 

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

 

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

 

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

 

mysql> select * from huanqiu.haha;

+----+-----------+

id | name      |

+----+-----------+

|  1 | 王士博    |

|  2 | 郭慧慧    |

|  3 | 周正      |

|  4 | 李敏      |

+----+-----------+

4 rows in set (0.00 sec)

 

 

2)默认情况下,vip是在master1上的。使用"ip addr"命令查看vip切换情况 

[root@master1 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:3c:25:42 brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.238/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/32 scope global eth0                              //这个32位子网掩码的vip地址表示该资源目前还在master1机器上

    inet 182.148.15.236/27 brd 82.48.115.255 scope global secondary eth0:0

    inet6 fe80::5054:ff:fe3c:2542/64 scope link

       valid_lft forever preferred_lft forever

 

停止master1机器上的mysql服务,根据配置中的脚本,mysql服务停了,keepalived也会停,从而vip资源将会切换到master2机器上。(mysql服务没有起来的时候,keepalived服务也无法顺利启动!)

[root@master1 ~]# /etc/init.d/mysql stop

Shutting down MySQL.. SUCCESS!

[root@master1 ~]# ps -ef|grep mysql

root     25812 21588  0 17:30 pts/0    00:00:00 grep mysql

[root@master1 ~]# ps -ef|grep keepalived

root     25814 21588  0 17:30 pts/0    00:00:00 grep keepalived

[root@master1 ~]# ip addr                  

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:3c:25:42 brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.238/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/27 brd 82.48.115.255 scope global secondary eth0:0

    inet6 fe80::5054:ff:fe3c:2542/64 scope link

       valid_lft forever preferred_lft forever

 

如上结果,发现32位子网掩码的vip没有了,说明此时vip资源已不在master1机器上了

查看下master1的系统日志,如下,会发现vip资源已经切换走了

[root@master1 ~]# tail -f /var/log/messages

Apr 15 17:17:43 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:17:48 localhost Keepalived_vrrp[23037]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:39 localhost Keepalived_healthcheckers[23036]: Stopped

Apr 15 17:30:39 localhost Keepalived_vrrp[23037]: VRRP_Instance(VI_1) sent 0 priority

Apr 15 17:30:39 localhost Keepalived_vrrp[23037]: VRRP_Instance(VI_1) removing protocol VIPs.

 

再到master2机器上,发现vip资源的确切换过来了

[root@master2 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:95:1f:6d brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.237/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/32 scope global eth0

    inet6 fe80::5054:ff:fe95:1f6d/64 scope link

       valid_lft forever preferred_lft forever

 

查看master2的系统日志

[root@master2 ~]# tail -f /var/log/messages

Apr 15 17:30:41 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:41 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:41 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:41 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

 

3)再次启动master1的mysql和keepalived服务。(注意:如果restart重启mysql,那么还要启动下keepalived,因为mysql重启,根据脚本会造成keepalived关闭)

注意:一定要先启动mysql服务,然后再启动keepalived服务。如果先启动keepalived服务,按照上面的配置,mysql没有起来,就会自动关闭keepalived。

[root@master1 ~]# /etc/init.d/mysql start

Starting MySQL.. SUCCESS!

 

[root@master1 ~]# /etc/init.d/keepalived start

正在启动 keepalived:                                      [确定]

 

启动这两个服务器后,稍微等过一会儿,注意观察会发现vip资源再次从master2机器上切换回来了。

[root@master1 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:3c:25:42 brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.238/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/32 scope global eth0

    inet 182.148.15.236/27 brd 82.48.115.255 scope global secondary eth0:0

    inet6 fe80::5054:ff:fe3c:2542/64 scope link

       valid_lft forever preferred_lft forever

 

[root@master1 ~]# tail -f /var/log/messages

Apr 15 17:40:41 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:41 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:41 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:41 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:46 localhost Keepalived_vrrp[27002]: Sending gratuitous ARP on eth0 for 182.148.15.236

 

再看看master2机器,发现vip资源又被恢复后的master1抢过去了

[root@master2 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:95:1f:6d brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.237/27 brd 182.148.15.255 scope global eth0

    inet6 fe80::5054:ff:fe95:1f6d/64 scope link

       valid_lft forever preferred_lft forever

 

[root@master2 ~]# tail -f /var/log/messages

Apr 15 17:30:41 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:30:46 localhost Keepalived_vrrp[8731]: Sending gratuitous ARP on eth0 for 182.148.15.236

Apr 15 17:40:41 localhost Keepalived_vrrp[8731]: VRRP_Instance(VI_1) Received advert with higher priority 101, ours 99

Apr 15 17:40:41 localhost Keepalived_vrrp[8731]: VRRP_Instance(VI_1) Entering BACKUP STATE

Apr 15 17:40:41 localhost Keepalived_vrrp[8731]: VRRP_Instance(VI_1) removing protocol VIPs.

 

4)同样,关闭master1机器的keepalived服务,vip资源会自动切换到master2机器上。当master1的keepalived服务恢复后,会将vip资源再次切回来。

[root@master1 ~]# /etc/init.d/keepalived stop

停止 keepalived:                                          [确定]

[root@master1 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:3c:25:42 brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.238/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/27 brd 82.48.115.255 scope global secondary eth0:0

    inet6 fe80::5054:ff:fe3c:2542/64 scope link

       valid_lft forever preferred_lft forever

 

查看master2,发现vip切过来了

[root@master2 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:95:1f:6d brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.237/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/32 scope global eth0

    inet6 fe80::5054:ff:fe95:1f6d/64 scope link

       valid_lft forever preferred_lft forever

 

再次恢复master1的keepalived服务,发现vip资源很快油切回来了。

[root@master1 ~]# /etc/init.d/keepalived start

正在启动 keepalived:                                      [确定]

[root@master1 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:3c:25:42 brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.238/27 brd 182.148.15.255 scope global eth0

    inet 182.148.15.236/32 scope global eth0

    inet 182.148.15.236/27 brd 82.48.115.255 scope global secondary eth0:0

    inet6 fe80::5054:ff:fe3c:2542/64 scope link

       valid_lft forever preferred_lft forever

 

在此查看master2,发现vip资源被切走了

[root@master2 ~]# ip addr

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 52:54:00:95:1f:6d brd ff:ff:ff:ff:ff:ff

    inet 182.148.15.237/27 brd 182.148.15.255 scope global eth0

    inet6 fe80::5054:ff:fe95:1f6d/64 scope link

       valid_lft forever preferred_lft forever

 

以上在vip资源切换过程中,对于客户端连接mysql(使用vip连接)来说几乎是没有任何影响的。

------------------------------------温馨提示(Keepalived的抢占和非抢占模式)---------------------------------------

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

keepalive是基于vrrp协议在linux主机上以守护进程方式,根据配置文件实现健康检查。

VRRP是一种选择协议,它可以把一个虚拟路由器的责任动态分配到局域网上的VRRP路由器中的一台。

控制虚拟路由器IP地址的VRRP路由器称为主路由器,它负责转发数据包到这些虚拟IP地址。

一旦主路由器不可用,这种选择过程就提供了动态的故障转移机制,这就允许虚拟路由器的IP地址可以作为终端主机的默认第一跳路由器。

 

keepalive通过组播,单播等方式(自定义),实现keepalive主备推选。工作模式分为抢占和非抢占(通过参数nopreempt来控制)。

1)抢占模式:

主服务正常工作时,虚拟IP会在主上,备不提供服务,当主服务优先级低于备的时候,备会自动抢占虚拟IP,这时,主不提供服务,备提供服务。

也就是说,工作在抢占模式下,不分主备,只管优先级。

 

如上配置,不管keepalived.conf里的state配置成master还是backup,只看谁的priority优先级高(一般而言,state为MASTER的优先级要高于BACKUP)。

priority优先级高的那一个在故障恢复后,会自动将VIP资源再次抢占回来!!

 

2)非抢占模式:

这种方式通过参数nopreempt(一般设置在advert_int的那一行下面)来控制。不管priority优先级,只要MASTER机器发生故障,VIP资源就会被切换到BACKUP上。

并且当MASTER机器恢复后,也不会去将VIP资源抢占回来,直至BACKUP机器发生故障时,才能自动切换回来。

 

千万注意:

nopreempt这个参数只能用于state为backup的情况,所以在配置的时候要把master和backup的state都设置成backup,这样才会实现keepalived的非抢占模式!

 

也就是说:

a)当state状态一个为master,一个为backup的时候,加不加nopreempt这个参数都是一样的效果。即都是根据priority优先级来决定谁抢占vip资源的,是抢占模式!

b)当state状态都设置成backup,如果不配置nopreempt参数,那么也是看priority优先级决定谁抢占vip资源,即也是抢占模式。

c)当state状态都设置成backup,如果配置nopreempt参数,那么就不会去考虑priority优先级了,是非抢占模式!即只有vip当前所在机器发生故障,另一台机器才能接管vip。即使优先级高的那一台机器恢复  后也不会主动抢回vip,只能等到对方发生故障,才会将vip切回来。

---------------------------------mysql状态检测脚本优化---------------------------------

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

上面的mysql监测脚本有点过于简单且粗暴,即脚本一旦监测到Master的mysql服务关闭,就立刻把keepalived服务关闭,从而实现vip转移!

 

下面对该脚本进行优化,优化后,当监测到Master的mysql服务关闭后,就会将vip切换到Backup上(但此时Master的keepalived服务不会被暴力kill

当Master的mysql服务恢复后,就会再次将VIP资源切回来!

 

[root@master ~]# cat /opt/chk_mysql.sh

#!/bin/bash

MYSQL=/usr/local/mysql/bin/mysql

MYSQL_HOST=localhost

MYSQL_USER=root

MYSQL_PASSWORD=123456

CHECK_TIME=3

  

#mysql  is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0

  

MYSQL_OK=1

  

function check_mysql_helth (){

    $MYSQL -h $MYSQL_HOST -u $MYSQL_USER -p${MYSQL_PASSWORD} -e "show status;" >/dev/null 2>&1

    if [ $? = 0 ] ;then

    MYSQL_OK=1

    else

    MYSQL_OK=0

    fi

    return $MYSQL_OK

}

while [ $CHECK_TIME -ne 0 ]

do

    let "CHECK_TIME -= 1"

    check_mysql_helth

if [ $MYSQL_OK = 1 ] ; then

    CHECK_TIME=0

    exit 0

fi

if [ $MYSQL_OK -eq 0 ] &&  [ $CHECK_TIME -eq 0 ]

then

    pkill keepalived

    exit 1

fi

sleep 1

done

 

你可能感兴趣的:(Mysql)