【原】ubuntu下Mysql的HA（corosync+pacemaker+drbd）

一、前提准备：

1、OS：ubuntu 12.04

2、cat /etc/hosts：

1 127.0.0.1 localhost

2 192.168.153.154 ha1

3 192.168.153.155 ha2

3、硬盘：/dev/sdb1

4、vip：192.168.153.200

二、corosync和pacemaker的安装配置：

1、[ha1+ha2]：

1 apt-get install -y pacemaker corosync

2、[ha1]：

1 corosync-keygen（时间较长）

2 scp /etc/corosync/authkey root@ha2:/etc/corosync/

3、[ha1+ha2]：取消corosync和pacemaker的开机启动：

 1 apt-get install -y sysv-rc-conf

 2 sysv-rc-conf（使用空格把drbd,pacemaker,corosync有X的地方取消掉，q退出）

 3 sed -i 's/127.0.0.1/192.168.153.0/g' /etc/corosync/corosync.conf

 4 sed -i 's/ver:       0/ver:       1/g' /etc/corosync/corosync.conf

 5 mkdir /var/log/cluster

 6 update-rc.d pacemaker start 50 1 2 3 4 5 . stop 01 0 6 .

 7 sed -i 's/no/yes/g' /etc/default/corosync

 8 service corosync start

 9 corosync-cfgtool -s

10 corosync-quorumtool -l

11 service pacemaker start

12 验证:crm status

13 

14 root@ha1:~# crm status

15 ============

16 Last updated: Tue Feb 25 22:18:44 2014

17 Last change: Tue Feb 25 22:17:49 2014 via crmd on ha1

18 Stack: openais

19 Current DC: ha1 - partition with quorum

20 Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c

21 2 Nodes configured, 2 expected votes

22 0 Resources configured.

23 ============

24 

25 

26 Online: [ ha1 ha2 ]

三、drbd的安装配置：

1、[ha1+ha2]：drbd的安装：

1 tar -zxvf drbd8_8.4.4.orig.tar.gz（由于drbd8.4.2有bug，所以这里选择手动安装8.4.4版本）

2 apt-get install -y debhelper flex xsltproc

3 cd drbd8_8.4.4

4 ./configure --localstatedir=/var --sysconfdir=/etc --with-km

5 make && make install

2、[ha1+ha2]：创建磁盘：

1 fdisk /dev/sdb

2 n

3 p

4 1

5 2048

6 ...

7 w

3、[ha1+ha2]：创建drbd的mysql资源：

 1 cat /etc/drbd.d/mysql.res

 2 resource mysql {

 3         device /dev/drbd0;

 4         disk /dev/sdb1;

 5         meta-disk internal;

 6         on ha1 {

 7                 address 192.168.153.154:7788;

 8         }

 9         on ha2 {

10                 address 192.168.153.155:7788;

11         }

12         syncer {

13                 rate 10M;

14         }

15 }

16 echo -e 'drbd # -> added' >> /etc/modules

17 drbdadm dump mysql

18 drbdadm create-md mysql

19 drbdadm up mysql

20 验证：drbd-overview

21 0:mysql/0  Connected Secondary/Secondary Inconsistent/Inconsistent C r----- 

4、[ha1]：

 1 drbdadm -- --overwrite-data-of-peer primary mysql

 2 cat /proc/drbd：

 3 root@ha1:~# cat /proc/drbd

 4 version: 8.4.4 (api:1/proto:86-101)

 5 GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@ha1, 2014-02-25 22:22:00

 6   0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----

 7     ns:108044 nr:0 dw:0 dr:108708 al:0 bm:6 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:20861776

 8 [>....................] sync'ed:  0.6% (20372/20476)M

 9 finish: 0:16:02 speed: 21,608 (21,608) K/sec

10 直到同步100%的时候才成功。结果是：

11 root@ha1:~# cat /proc/drbd

12 version: 8.4.4 (api:1/proto:86-101)

13 GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@ha1, 2014-02-25 22:22:00

14   0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----

15     ns:20969820 nr:0 dw:0 dr:20970484 al:0 bm:1280 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

16 service drbd restart

17 mkfs.ext4 /dev/drbd0

18 mkdir /mydata

19 mount /dev/drbd0 /mydata

20 mkdir -p /mydata/testfile  

21 umount /dev/drbd0  

22 drbdadm secondary mysql

5、[ha2]：

1 service drbd restart

2 drbdadm primary mysql

3 4 mkdir /mydata

5 mount /dev/drbd0 /mydata

6 ll /mydata/ 如果可以看到testfile说明成功。

7 umount /dev/drbd0 

8 drbdadm secondary mysql（如果中间出现错误，可以把ha1和ha2的drbd重启即可。）

6、[ha1]:

1 drbdadm primary mysql

7、[ha1+ha2]：mysql的安装配置

1 apt-get install -y mysql-server

2 sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf

3 service mysql restart

4 mysql -uroot -p

5 grant all privileges on *.* to root@"%" identified by "password" with grant option;

6 FLUSH PRIVILEGES;

7 delete from mysql.user where user="";

8 quit;

9 service mysql stop

8、[ha1]: 

1 drbdadm primary mysql

2 drbd-overview

3 mount /dev/drbd0 /mydata

4 cp -R -p /var/lib/mysql /mydata/

9、[ha1+ha2]: 

1 sed -i 's/2345/!0123456/g' /etc/init/mysql.conf

2 sed -i -e '/datadir/cdatadir         = \/mydata\/mysql'  /etc/mysql/my.cnf

3 sed -i '/\/var\/lib\/mysql\/ /c\/mydata\/mysql\/ r,' /etc/apparmor.d/usr.sbin.mysqld

4 sed -i '/\/var\/lib\/mysql\/\*\* rwk/c\/mydata\/mysql\/** rwk,' /etc/apparmor.d/usr.sbin.mysqld

10、[ha1]:

1 service mysql start

2 mysql -uroot -p

3 CREATE DATABASE test1;

4 quit

5 service mysql stop

6 umount /dev/drbd0

7 drbdadm secondary mysql

11、[ha2]:

1 drbdadm primary mysql

2 mount /dev/drbd0 /mydata/

3 service mysql start

4 查看是否有数据库test1

5 service mysql stop

6 umount /dev/drbd0

7 drbdadm secondary mysql

12、[ha1]: 

1 drbdadm primary mysql

13、[ha1+ha2]: 

1 service drbd stop

14、[ha1]:

 1 crm configure property stonith-enabled=false

 2 crm configure property no-quorum-policy=ignore

 3 

 4 crm configure

 5 primitive mysqldrbd ocf:heartbeat:drbd params drbd_resource=mysql op start timeout=240 op stop timeout=100 op monitor role=Master interval=20 timeout=30 op monitor role=Slave interval=30 timeout=30

 6 ms ms_mysqldrbd mysqldrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true

 7 commit

 8 exit

 9 验证:crm status

10 root@ha1:~# crm status

11 ============

12 Last updated: Tue Feb 25 23:06:09 2014

13 Last change: Tue Feb 25 23:06:05 2014 via cibadmin on ha1

14 Stack: openais

15 Current DC: ha1 - partition with quorum

16 Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c

17 2 Nodes configured, 2 expected votes

18 2 Resources configured.

19 ============

20 

21 

22 Online: [ ha1 ha2 ]

23 

24 

25   Master/Slave Set: ms_mysqldrbd [mysqldrbd]

26      Masters: [ ha1 ]

27       Slaves: [ ha2 ]

28 

29 crm configure

30 primitive mystore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/mydata fstype=ext4 op start timeout=60 op stop timeout=60

31 colocation mystore_with_ms_mysqldrbd inf: mystore ms_mysqldrbd:Master

32 order mystore_after_ms_mysqldrbd mandatory: ms_mysqldrbd:promote mystore:start

33 commit

34 exit

35 验证:crm status,mount

36   Master/Slave Set: ms_mysqldrbd [mysqldrbd]

37       Masters: [ ha1 ]

38       Slaves: [ ha2 ]

39   mystore (ocf::heartbeat:Filesystem): Started ha1

40 

41 

42 crm configure

43 primitive mysql lsb:mysql

44 colocation mysql_with_mystore inf: mysql mystore

45 order mysql_after_mystore mandatory: mystore mysql

46 commit

47 exit

48 验证:crm status

49 Master/Slave Set: ms_mysqldrbd [mysqldrbd]

50       Masters: [ ha1 ]

51       Slaves: [ ha2 ]

52   mystore (ocf::heartbeat:Filesystem): Started ha1

53   mysql (lsb:mysql): Started ha1

54 

55 crm configure

56 primitive vip ocf:heartbeat:IPaddr params ip=192.168.153.200 nic=eth0 cidr_netmask=255.255.255.0  

57 colocation vip_with_ms_mysqldrbd inf: ms_mysqldrbd:Master vip 

58 commit

59 exit

60 验证crm status

61 Master/Slave Set: ms_mysqldrbd [mysqldrbd]

62       Masters: [ ha1 ]

63       Slaves: [ ha2 ]

64   mystore (ocf::heartbeat:Filesystem): Started ha1

65   mysql (lsb:mysql): Started ha1

66   vip (ocf::heartbeat:IPaddr): Started ha1

四、验证：

1、客户端mysql验证：

 1 mysql -uroot -p -h 192.168.153.200  

 2 mysql> show databases;

 3 +--------------------+

 4 | Database           |

 5 +--------------------+

 6 | information_schema |

 7 | mysql              |

 8 | performance_schema |

 9 | test               |

10 | test1              |

11 +--------------------+

2、模拟故障：

 1 root@ha1:~# crm

 2 crm(live)# node

 3 crm(live)node# standby

 4 crm(live)node# show

 5 ha1: normal

 6 standby: on

 7 ha2: normal

 8 crm(live)node# exit

 9 bye

10 此时看一下状态：

11 root@ha1:~# crm status

12 Node ha1: standby

13 Online: [ ha2 ]

14 

15 

16  Master/Slave Set: ms_mysqldrbd [mysqldrbd]

17      Masters: [ ha2 ]

18      Stopped: [ mysqldrbd:0 ]

19  mystore (ocf::heartbeat:Filesystem): Started ha2

20  mysql (lsb:mysql): Started ha2

21  vip (ocf::heartbeat:IPaddr): Started ha2

此时，mysql已经运行于ha2上了。