heartbeat+drbd+mysql构建mysql高可用群集

heartbeat+drbd+mysql构建mysql高可用群集

1. 试验环境：

操作系统：Red Hat Enterprise Linux 5.4

所需的软件包：

mysql-5.5.22.tar.gz //mysql绿色安装包

httpd-2.2.19.tar.bz2 //apache源码包

php-5.3.7.tar.bz2 //php源码包

phpMyAdmin-2.11.10.1-all-languages.zip //mysql图形界面管理工具包

drbd安装包

drbd83-8.3.8-1.el5.centos.i386.rpm

kmod-drbd83-8.3.8-1.el5.centos.i686.rpm

heartbeat安装包

heartbeat-2.1.4-9.el5.i386.rpm

heartbeat-devel-2.1.4-9.el5.i386.rpm

heartbeat-pils-2.1.4-10.el5.i386.rpm

heartbeat-stonith-2.1.4-10.el5.i386.rpm

libnet-1.1.4-3.el5.i386.rpm

perl-MailTools-1.77-1.el5.noarch.rpm

实验拓扑：

 

 

主从节点的基本环境配置：

node1.ws.cn 网卡1：192.168.2.10 心跳线：172.16.10.1 主节点

node2.ws.cn 网卡1：192.168.2.20 心跳线：172.16.10.2 从节点

修改主机名：

vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=node1.ws.cn

节点2则设置为node2.ws.cn

修改hosts文件

为了实验方便这里不再安装DNS服务器，只是更改hosts文件

[root@node1 ~]# cat /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

192.168.2.10 node1.ws.cn

192.168.2.20 node2.ws.cn

节点2与节点1同样的设置不再重复

增加一块新的磁盘用来做冗余备份

[root@node1 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 261 2096451 83 Linux

/dev/sda2 262 656 3172837+ 82 Linux swap / Solaris

/dev/sda3 657 787 1052257+ 83 Linux

/dev/sda4 788 2610 14643247+ 5 Extended

/dev/sda5 788 2610 14643216 83 Linux

Disk /dev/sdb: 5368 MB, 5368709120 bytes

255 heads, 63 sectors/track, 652 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

[root@node1 ~]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdb: 5368 MB, 5368709120 bytes

255 heads, 63 sectors/track, 652 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4):

Value out of range.

Partition number (1-4):

Value out of range.

Partition number (1-4): 1

First cylinder (1-652, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-652, default 652):

Using default value 652

Command (m for help): p

Disk /dev/sdb: 5368 MB, 5368709120 bytes

255 heads, 63 sectors/track, 652 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 1 652 5237158+ 83 Linux

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@node1 ~]#

在主从节点上执行： hwclock -s 同步节点时区

下面用源码来搭建lamp环境，用于操作管理mysql

为了使实验的顺利进行，首先检查系统是否安装了apache，php及mysql的软件包，

Rpm –qa 服务名称（例：如apache，rpm –qa httpd）

如果安装了就将其卸载： rpm –e 服务名称

也可以用yum来卸载，可以很好的解决依赖性关系

在主从服务器上

配置mysql-5.5.22.tar.gz 的绿色安装包

解压压缩包

Tar -zxvf mysql-5.5.15-linux2.6-i686.tar.gz -C /usr/local/

进入目录：

cd /usr/local/

为了以后的操作方便添加以下连接

ln -s mysql-5.5.15-linux2.6-i686 mysql

 

 

进入mysql主目录

cd mysql

查看主目录下的所有文件

ll

[root@node1 mysql]# ll

总计 76

drwxr-xr-x 2 root root 4096 05-18 15:19 bin

-rw-r--r-- 1 7161 wheel 17987 2011-07-14 COPYING

drwxr-xr-x 4 root root 4096 05-18 15:21 data

drwxr-xr-x 2 root root 4096 05-18 15:22 docs

drwxr-xr-x 3 root root 4096 05-18 15:21 include

-rw-r--r-- 1 7161 wheel 7604 2011-07-14 INSTALL-BINARY

drwxr-xr-x 3 root root 4096 05-18 15:22 lib

drwxr-xr-x 4 root root 4096 05-18 15:19 man

drwxr-xr-x 10 root root 4096 05-18 15:22 mysql-test

-rw-r--r-- 1 7161 wheel 2552 2011-07-14 README

drwxr-xr-x 2 root root 4096 05-18 15:22 scripts

drwxr-xr-x 27 root root 4096 05-18 15:21 share

drwxr-xr-x 4 root root 4096 05-18 15:22 sql-bench

drwxr-xr-x 2 root root 4096 05-18 15:22 support-files

添加mysql用户组

Groupadd mysql

在mysql组内添加一个mysql用户

Useradd –r -g mysql mysql

更改当前目录下的所有文件的所有者和所属组

Chown –R mysql .

Chgrp –R mysql .

更改后的文件属性

[root@node1 mysql]# ll

总计 76

drwxr-xr-x 2 mysql mysql 4096 05-18 15:19 bin

-rw-r--r-- 1 mysql mysql 17987 2011-07-14 COPYING

drwxr-xr-x 4 mysql mysql 4096 05-18 15:21 data

drwxr-xr-x 2 mysql mysql 4096 05-18 15:22 docs

drwxr-xr-x 3 mysql mysql 4096 05-18 15:21 include

-rw-r--r-- 1 mysql mysql 7604 2011-07-14 INSTALL-BINARY

drwxr-xr-x 3 mysql mysql 4096 05-18 15:22 lib

drwxr-xr-x 4 mysql mysql 4096 05-18 15:19 man

drwxr-xr-x 10 mysql mysql 4096 05-18 15:22 mysql-test

-rw-r--r-- 1 mysql mysql 2552 2011-07-14 README

drwxr-xr-x 2 mysql mysql 4096 05-18 15:22 scripts

drwxr-xr-x 27 mysql mysql 4096 05-18 15:21 share

drwxr-xr-x 4 mysql mysql 4096 05-18 15:22 sql-bench

drwxr-xr-x 2 mysql mysql 4096 05-18 15:22 support-files

Scripts/mysql_install_db --user=mysql

Chown –R root .

Chown –R mysql data

[root@node1 mysql]# ll

总计 76

drwxr-xr-x 2 root mysql 4096 05-18 15:19 bin

-rw-r--r-- 1 root mysql 17987 2011-07-14 COPYING

drwxr-xr-x 5 mysql mysql 4096 05-18 15:31 data

drwxr-xr-x 2 root mysql 4096 05-18 15:22 docs

drwxr-xr-x 3 root mysql 4096 05-18 15:21 include

-rw-r--r-- 1 root mysql 7604 2011-07-14 INSTALL-BINARY

drwxr-xr-x 3 root mysql 4096 05-18 15:22 lib

drwxr-xr-x 4 root mysql 4096 05-18 15:19 man

drwxr-xr-x 10 root mysql 4096 05-18 15:22 mysql-test

-rw-r--r-- 1 root mysql 2552 2011-07-14 README

drwxr-xr-x 2 root mysql 4096 05-18 15:22 scripts

drwxr-xr-x 27 root mysql 4096 05-18 15:21 share

drwxr-xr-x 4 root mysql 4096 05-18 15:22 sql-bench

drwxr-xr-x 2 root mysql 4096 05-18 15:22 support-files

添加mysql的环境变量

Vim /etc/profile

 

 

执行 . /etc/profile 重新读取环境变量使配置生效

拷贝启动文件

cp -p support-files/mysql.server /etc/init.d/mysqld

拷贝主配置文件

cp -p support-files/my-medium.cnf /etc/my.cnf

加入到启动脚本

chkconfig --add mysqld

配置头文件：

cd /etc/ld.so.conf.d/

编辑新文件 mysql.conf

加入：/usr/local/mysql/lib

配置库文件：

cd /usr/include/

ln -s /usr/local/mysql/include mysql

[root@node1 include]# ll |grep mysql

lrwxrwxrwx 1 root root 24 05-18 15:46 mysql -> /usr/local/mysql/include

启动mysql服务

[root@node1 ld.so.conf.d]# service mysqld start

Starting MySQL..... [确定]

查看监听端口

[root@node1 ld.so.conf.d]# netstat -tupln |grep mysqld

tcp 0 0 :::3306 :::* LISTEN 3010/mysqld

[root@node1 ld.so.conf.d]#

关闭服务

[root@node1 ld.so.conf.d]# service mysqld stop

Shutting down MySQL. [确定]

查看服务器的启动状态

[root@node1 ld.so.conf.d]# chkconfig --list |grep mysqld

mysqld 0:关闭 1:关闭 2:启用 3:启用 4:启用 5:启用 6:关闭

由于后面我们还会做heartbeat服务，用heartbeat服务来管理mysql这里设置为在何种级别下都不会自动启动

[root@node1 ld.so.conf.d]# chkconfig mysqld off

[root@node1 ld.so.conf.d]# chkconfig --list |grep mysqld

mysqld 0:关闭 1:关闭 2:关闭 3:关闭 4:关闭 5:关闭 6:关闭

[root@node1 ld.so.conf.d]#

Mysql配置完毕

安装并配置httpd-2.2.19.tar.bz2源码压缩包

。

1. Tar -jxvf httpd-2.2.19.tar.bz2 -C /usr/local/src //解压缩

2.Cd /usr/local/src/httpd-2.2.19/ //进入httpd-2.2.19的主目录

3. ./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd --enable-ssl --with-z --enable-so //配置

4. Make //编译

5. Make install //安装

6.配置启动项。

vim /etc/profile

 

 

. /etc/profile //重新读取文件，使变量生效

查看

[root@node1 httpd-2.2.19]# echo $PATH

/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin:/usr/local/mysql/bin:/usr/local/apache/bin

Mysql变量 ，apache变量

配置头文件

Cd /etc/ld.so.conf.d/

Vim httpd.conf

插入：/usr/local/apache/bin

Ldconfig -v //重新加载头文件

配置库文件

Cd /usr/include/

Ln -s /usr/local/apache/include httpd

查看

[root@node1 include]# ll |grep httpd

lrwxrwxrwx 1 root root 25 05-18 23:51 httpd -> /usr/local/apache/include

[root@node1 include]#

添加启动项：

cd /etc/init.d/

cp /usr/local/apache/bin/apachectl /etc/init.d/httpd

启动apache

service httpd start

查看监听端口：

[root@node1 include]# netstat -tupln |grep httpd

tcp 0 0 :::80 :::* LISTEN 30070/httpd

安装并配置php-5.3.7.tar.bz2源码压缩包

。

Tar -jxvf php-5.3.7.tar.bz2 -C /usr/local/src //解压缩

Cd /usr/local/src/php-5.3.7/ //切换目录

./configure --prefix=/usr/local/php --with-apxs2=/usr/local/apache/bin/apxs --with-mysql=/usr/local/mysql --with-mysqli=/usr/local/mysql/bin/mysql_config --enable-mbstring=all //配置

Make //编译

Make install //安装

编辑主配置文件

vim /etc/httpd/httpd.conf

 

 

 

 

手工编译安装php后，如果一切顺利解决了的话，重启apache时，libphp5.so: cannot restore segment prot after reloc: Permission denied 错误：如果你是开着seliunx 的话，运行下面的命令： chcon -t texrel_shlib_t /usr/local/apache/modules/libphp5.so <具体chcon 的命令的意思，可以自己man一下，它是selinux有关的命令，更改文件的标签。ls命令在命令后加个 －Z 或者加 –context 查看标签> 如果 没有开selinux 的话，找到libphp5.so，然后chmod 777 libphp5.so 源自：http://www.phpfreaks.com/forums/index.php?topic=133131.0;wap2

重启apache服务确保更新的配置都能起作用

service httpd restart

测试：

Mv index.html index.php

Cd /usr/local/apache/htdocs/

Vim index.php

 

 

结果：

 

 

测试 php 与 mysql 的连接

修改网页文件如下：

Vim index.php

 

结果：

 

基本环境搭建完成

命令行登录如下：

[root@node1 Server]# /usr/local/mysql/bin/mysql -u root

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.5.15-log MySQL Community Server (GPL)

Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

mysql>

mysql> show databases;

+--------------------+

| Database |

+--------------------+

| information_schema |

| mysql |

| performance_schema |

| test |

+--------------------+

4 rows in set (0.22 sec)

mysql>

 

添加web管理平台

往往为了管理的方便还可以加载一个phpmyadmin管理平台

解压缩包

unzip -xvf phpMyAdmin-2.11.10.1-all-languages.zip

更改包名并移动到主站点下

mv phpMyAdmin-2.11.10.1-all-languages /usr/local/apache/htdocs/

在本地浏览器输入：http://localhost/phpmyadmin

 

 

或者在远端浏览器输入：http://192.168.2.10/phpmyadmin

 

即可以网页的形式进行访问操作

在主从节点上配置drbd

安装软件包

rpm -ivh drbd83-8.3.8-1.el5.centos.i386.rpm

rpm -ivh kmod-drbd83-8.3.8-1.el5.centos.i686.rpm

拷贝主配置文档

cp /usr/share/doc/drbd83-8.3.8/drbd.conf /etc/

如果：/etc/下有drbd.conf替换即可

编辑管理文档：

cp /etc/drbd.d/global_common.conf /etc/drbd.d/global_common.conf.back //编辑之前备份一份

vim /etc/drbd.d/global_common.conf

common {

protocol C;

startup {

wfc-timeout 120;

degr-wfc-timeout 120;

}

disk {

on-io-error detach;

fencing resource-only;

}

net {

cram-hmac-alg "shal";

shared-secret "mydrbdlab";

}

syncer {

rate 100M;

}

}

定义资源：

vim /etc/drbd.d/mysql.res

resource mysql {

on node1.ws.cn{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.2.10:7789;

meta-disk internal;

}

on node2.ws.cn{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.2.20:7789;

meta-disk internal;

}

}

初始化数据：

drbdadm create-md mysql

service drbd start //启动服务

主从服务器需要同时启动

chkconfig drbd on //设置开机自起

查看状态

cat /proc/drbd

在主节点上执行

drbdadm -- --overwrite-data-of-peer primary mysql

创建文件系统（在主节点上实现）

mkfs -t ext3 -L drbdmysql /dev/drbd0

mkdir /mnt/mysql //创建挂载点

mount /dev/drbd0 /mnt/mysql //挂载设备

echo "good" > /mnt/mysql/index.html //创建文件

测试，把node1变成从的，node2 变成住的

在节点node1上

umount /mnt/mysql //卸载设备

drbdadm secondary mysql //更改为辅助设备

drbdadm role mysql //查看状态

Secondary/Secondary

在node2 节点上，

mkdir /mnt/mysql //创建挂载点

drbdadm primary mysql //更改设备为主设备

drbd-overview //查看状态

0:web Connected Primary/Secondary UpToDate/UpToDate C r----

drbdadm role mysql

Primary/Secondary

mount /dev/drbd0 /mnt/mysql

ll /mnt/mysql

total 20

-rw-r--r-- 1 root root 6 May 7 00:46 index.html

drwx------ 2 root root 16384 May 7 00:45 lost+found

cd /mnt/mysql

vim test.html

umount /mnt/mysql

drbdadm secondary mysql

切换到node1上

drbdadm primary mysql

mount /dev/drbd0 /mnt/mysql

ll /mnt/mysql

total 24

-rw-r--r-- 1 root root 6 May 7 00:46 index.html

drwx------ 2 root root 16384 May 7 00:45 lost+found

-rw-r--r-- 1 root root 13 May 7 00:58 test.html

Drbd 配置完成

在主从节点上安装heartbeat服务

:

安装软件包：

为了解决依赖性关系这里使用yum安装（在此之前yum厂库已配好）

yum localinstall heartbeat-2.1.4-9.el5.i386.rpm heartbeat-devel-2.1.4-9.el5.i386.rpm heartbeat-pils-2.1.4-10.el5.i386.rpm heartbeat-stonith-2.1.4-10.el5.i386.rpm libnet-1.1.4-3.el5.i386.rpm perl-MailTools-1.77-1.el5.noarch.rpm –nogpgcheck      //本地安装，不进行pgp检查

进入heartbeat主目录

Cd /etc/ha.d/

拷贝三个配置文档

cp -p /usr/share/doc/heartbeat-2.1.4/ha.cf ./

cp -p /usr/share/doc/heartbeat-2.1.4/haresources ./

cp -p /usr/share/doc/heartbeat-2.1.4/authkeys ./

编辑三个文档：

修改主配置文档

Vim ha.cf

…………………………………………….省略

#bcast eth1 eth2 # Linux

#bcast le0 # Solaris

#bcast le1 le2 # Solaris

bcast eth1 心跳线 # Linux

#

# Set up a multicast heartbeat medium

……………………………………………….省略

#node ken3

#node kathy

node node1.ws.cn //节点名称

node node2.ws.cn

#

# Less common options...

……………………………………………….省略

注释：以下各项可以根据具体情况做调整

logfacility local0       //定义日志，使用syslog；

keepalive 2           //定义心跳频率；

deadtime 15          //如果其他节点15S内没有回应，则确认其死 亡；

warntime 5           //确认一个节点连接不上5S之后将警告信息写入日志；

initdead 120          //在其他节点死掉之后，系统启动前需要等待的时间，一般为deadtime的两倍；

udpport 694          // 用来通信的端口；

watchdog /dev/watchdog  //可选项

auto_failback on        // 设置当死亡节点恢复正常之后是否重新启用；容易发生数据不一致的情况，必须项，不然后面hb_standby命令无法使用；

Vim haresources //定义资源

……………………………………………….省略

#node-name resource1 resource2 ... resourceN

node1.ws.cn drbddisk Filesystem::/dev/drbd0::/mysql::ext3 mysqld httpd 192.168.2.100/24/eth0/192.168.2.255  //定义Heartbeat需要管理的资源，并且每一部分在/etc/ha.d/haresources.d下有对应的脚本；如果主节点失效，备节点将通过执行这些脚本来启用资源；如果脚本有参数可以使用“：：”来进行分割；

#

……………………………………………….省略

Node1.ws.cn //主节点名称

filesystem 管理文件系统资源的脚本，后面参数定义drbd设备，挂载点，文件系统类型

mysqld  管理mysql的脚本，需要从/etc/init.d/mysqld 拷贝

httpd  管理apache的脚本，需要从/etc/init.d/httpd 拷贝

192.168.43.20  //VIP,用来连接MySQL的IP，当Heartbeat启用之后会自动分配；

查看 resource.d/内的脚本

[root@node1 resource.d]# ll

-rwxr-xr-x 1 root root    828 2009-07-25 apache
-rwxr-xr-x 1 root root    805 2009-07-25 AudibleAlarm
-rwxr-xr-x 1 root root    760 2009-07-25 db2
-rwxr-xr-x 1 root root    910 2009-07-25 Delay
-rwxr-xr-x 1 root root   2141 2010-06-04 drbddisk
-rwxr-xr-x 1 root root   1167 2010-06-04 drbdupper
-rwxr-xr-x 1 root root   1903 2009-07-25 Filesystem
-rwxr-xr-x 1 root root   2329 2009-07-25 hto-mapfuncs
-rwxr-xr-x 1 root root   3426 05-18 23:54 httpd
-rwxr-xr-x 1 root root    951 2009-07-25 ICP
-rwxr-xr-x 1 root root   3424 2009-07-25 ids
-rwxr-xr-x 1 root root   2273 2009-07-25 IPaddr
-rwxr-xr-x 1 root root   1825 2009-07-25 IPaddr2
-rwxr-xr-x 1 root root   1391 2009-07-25 IPsrcaddr
-rwxr-xr-x 1 root root    622 2009-07-25 IPv6addr
lrwxrwxrwx 1 root root     20 05-13 19:17 ldirectord -> /usr/sbin/ldirectord
-rwxr-xr-x 1 root root   1091 2009-07-25 LinuxSCSI
-rwxr-xr-x 1 root root    790 2009-07-25 LVM
-rwxr-xr-x 1 root root   4137 2009-07-25 LVSSyncDaemonSwap
-rwxr-xr-x 1 root root   1125 2009-07-25 MailTo
-rwxr-xr-x 1 root mysql 10650 2011-07-14 mysqld
-rwxr-xr-x 1 root root   2923 2009-07-25 OCF
-rwxr-xr-x 1 root root    739 2009-07-25 portblock
-rwxr-xr-x 1 root root   1160 2009-07-25 Raid1
-rwxr-xr-x 1 root root   1563 2009-07-25 SendArp
-rwxr-xr-x 1 root root   1012 2009-07-25 ServeRAID
-rwxr-xr-x 1 root root   1294 2009-07-25 WAS
-rwxr-xr-x 1 root root   1166 2009-07-25 WinPopup
-rwxr-xr-x 1 root root    666 2009-07-25 Xinetd

 

Vim authkeys //设置节点之间的通讯密匙

……………………………………………….省略

#auth 1

#1 crc

#2 sha1 HI!

#3 md5 Hello!

auth 2

2 md5 488ddcf258a1ac2508c2f7f624300cf2

# 488ddcf258a1ac2508c2f7f624300cf2 为随机数【使用命令：

[root@node1 ~]# dd if=/dev/random bs=512 count=1 |openssl md5

0+1 records in

0+1 records out

128 bytes (128 B) copied, 0.000679603 seconds, 188 kB/s

488ddcf258a1ac2508c2f7f624300cf2

[root@node1 ~]#

生成】

为了安全需要更改密匙存放文件的权限

chmod 600 authkeys

启动heartbeat服务

Service heartbeat start

查看主节点日志如下：

[root@node1 ~]# tail -f /var/log/messages

May 19 13:58:28 node1 kernel: block drbd0: peer( Primary -> Secondary )

May 19 13:58:28 node1 heartbeat: [6391]: info: standby: acquire [foreign] resources from node2.ws.cn

May 19 13:58:29 node1 heartbeat: [6420]: info: acquire local HA resources (standby).

May 19 13:58:29 node1 ResourceManager[6435]: info: Acquiring resource group: node1.ws.cn drbddisk Filesystem::/dev/drbd0::/mnt/mysql::ext3 mysqld httpd 192.168.2.100/24/eth0/192.168.2.255

May 19 13:58:29 node1 ResourceManager[6435]: info: Running /etc/ha.d/resource.d/drbddisk start

May 19 13:58:29 node1 kernel: block drbd0: role( Secondary -> Primary )

May 19 13:58:30 node1 Filesystem[6493]: INFO: Resource is stopped

May 19 13:58:30 node1 ResourceManager[6435]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /mnt/mysql ext3 start

May 19 13:58:30 node1 ipfail: [6410]: info: Ping node count is balanced.

May 19 13:58:30 node1 Filesystem[6574]: INFO: Running start for /dev/drbd0 on /mnt/mysql

May 19 13:58:31 node1 kernel: kjournald starting. Commit interval 5 seconds

May 19 13:58:31 node1 kernel: EXT3 FS on drbd0, internal journal

May 19 13:58:31 node1 kernel: EXT3-fs: mounted filesystem with ordered data mode.

May 19 13:58:31 node1 Filesystem[6563]: INFO: Success

May 19 13:58:31 node1 ResourceManager[6435]: info: Running /etc/ha.d/resource.d/mysqld start

May 19 13:58:36 node1 ResourceManager[6435]: info: Running /etc/ha.d/resource.d/httpd start

May 19 13:58:38 node1 IPaddr[7003]: INFO: Resource is stopped

May 19 13:58:38 node1 ResourceManager[6435]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.2.100/24/eth0/192.168.2.255 start

May 19 13:58:39 node1 IPaddr[7109]: INFO: Using calculated netmask for 192.168.2.100: 255.255.255.0

May 19 13:58:39 node1 IPaddr[7109]: INFO: eval ifconfig eth0:0 192.168.2.100 netmask 255.255.255.0 broadcast 192.168.2.255

May 19 13:58:39 node1 avahi-daemon[4111]: Registering new address record for 192.168.2.100 on eth0.

May 19 13:58:40 node1 IPaddr[7080]: INFO: Success

May 19 13:58:40 node1 heartbeat: [6420]: info: local HA resource acquisition completed (standby).

May 19 13:58:40 node1 heartbeat: [6391]: info: Standby resource acquisition done [foreign].

May 19 13:58:40 node1 heartbeat: [6391]: info: Initial resource acquisition complete (auto_failback)

May 19 13:58:41 node1 heartbeat: [6391]: info: remote resource transition completed.

从节点日志：

[root@node2 ~]# tail -f /var/log/messages

May 19 13:58:28 node2 ResourceManager[6024]: info: Running /etc/ha.d/resource.d/drbddisk stop

May 19 13:58:28 node2 kernel: block drbd0: role( Primary -> Secondary )

May 19 13:58:28 node2 heartbeat: [6010]: info: foreign HA resource release completed (standby).

May 19 13:58:28 node2 heartbeat: [3954]: info: Local standby process completed [foreign].

May 19 13:58:29 node2 kernel: block drbd0: peer( Secondary -> Primary )

May 19 13:58:30 node2 ipfail: [4040]: info: No giveup timer to abort.

May 19 13:58:40 node2 heartbeat: [3954]: WARN: 1 lost packet(s) for [node1.ws.cn] [20:22]

May 19 13:58:40 node2 heartbeat: [3954]: info: remote resource transition completed.

May 19 13:58:40 node2 heartbeat: [3954]: info: No pkts missing from node1.ws.cn!

May 19 13:58:40 node2 heartbeat: [3954]: info: Other node completed standby takeover of foreign resources.

在主节点上：

虚拟ip已启动：

 

磁盘挂载情况：

 

外网访问结果：

 

当主节点node1.ws.cn当机时

主节点日志：

[root@node1 ~]# tail -f /var/log/messages

May 19 13:58:39 node1 IPaddr[7109]: INFO: Using calculated netmask for 192.168.2.100: 255.255.255.0

May 19 13:58:39 node1 IPaddr[7109]: INFO: eval ifconfig eth0:0 192.168.2.100 netmask 255.255.255.0 broadcast 192.168.2.255

May 19 13:58:39 node1 avahi-daemon[4111]: Registering new address record for 192.168.2.100 on eth0.

May 19 13:58:40 node1 IPaddr[7080]: INFO: Success

May 19 13:58:40 node1 heartbeat: [6420]: info: local HA resource acquisition completed (standby).

May 19 13:58:40 node1 heartbeat: [6391]: info: Standby resource acquisition done [foreign].

May 19 13:58:40 node1 heartbeat: [6391]: info: Initial resource acquisition complete (auto_failback)

May 19 13:58:41 node1 heartbeat: [6391]: info: remote resource transition completed.

May 19 14:10:04 node1 avahi-daemon[4111]: Invalid query packet.

May 19 14:12:37 node1 last message repeated 9 times

May 19 14:12:40 node1 last message repeated 8 times

May 19 14:27:12 node1 heartbeat: [6391]: info: node1.ws.cn wants to go standby [all]

May 19 14:27:12 node1 heartbeat: [6391]: info: standby: node2.ws.cn can take our all resources

May 19 14:27:12 node1 heartbeat: [7914]: info: give up all HA resources (standby).

May 19 14:27:13 node1 ResourceManager[7927]: info: Releasing resource group: node1.ws.cn drbddisk Filesystem::/dev/drbd0::/mnt/mysql::ext3 mysqld httpd 192.168.2.100/24/eth0/192.168.2.255

May 19 14:27:13 node1 ResourceManager[7927]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.2.100/24/eth0/192.168.2.255 stop

May 19 14:27:14 node1 IPaddr[7996]: INFO: ifconfig eth0:0 down

May 19 14:27:14 node1 avahi-daemon[4111]: Withdrawing address record for 192.168.2.100 on eth0.

May 19 14:27:14 node1 IPaddr[7967]: INFO: Success

May 19 14:27:14 node1 ResourceManager[7927]: info: Running /etc/ha.d/resource.d/httpd stop

May 19 14:27:15 node1 ResourceManager[7927]: info: Running /etc/ha.d/resource.d/mysqld stop

May 19 14:27:16 node1 ResourceManager[7927]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /mnt/mysql ext3 stop

May 19 14:27:17 node1 Filesystem[8116]: INFO: Running stop for /dev/drbd0 on /mnt/mysql

May 19 14:27:17 node1 Filesystem[8116]: INFO: Trying to unmount /mnt/mysql

May 19 14:27:17 node1 Filesystem[8116]: INFO: unmounted /mnt/mysql successfully

May 19 14:27:18 node1 Filesystem[8105]: INFO: Success

May 19 14:27:18 node1 ResourceManager[7927]: info: Running /etc/ha.d/resource.d/drbddisk stop

May 19 14:27:18 node1 kernel: block drbd0: role( Primary -> Secondary )

May 19 14:27:18 node1 heartbeat: [7914]: info: all HA resource release completed (standby).

May 19 14:27:18 node1 heartbeat: [6391]: info: Local standby process completed [all].

May 19 14:27:19 node1 kernel: block drbd0: peer( Secondary -> Primary )

从节点日志：

[root@node2 ~]# tail -f /var/log/messages

May 19 14:27:12 node2 heartbeat: [3954]: info: node1.ws.cn wants to go standby [all]

May 19 14:27:18 node2 kernel: block drbd0: peer( Primary -> Secondary )

May 19 14:27:18 node2 heartbeat: [3954]: info: standby: acquire [all] resources from node1.ws.cn

May 19 14:27:19 node2 heartbeat: [6849]: info: acquire all HA resources (standby).

May 19 14:27:19 node2 ResourceManager[6862]: info: Acquiring resource group: node1.ws.cn drbddisk Filesystem::/dev/drbd0::/mnt/mysql::ext3 mysqld httpd 192.168.2.100/24/eth0/192.168.2.255

May 19 14:27:19 node2 ResourceManager[6862]: info: Running /etc/ha.d/resource.d/drbddisk start

May 19 14:27:19 node2 kernel: block drbd0: role( Secondary -> Primary )

May 19 14:27:20 node2 Filesystem[6921]: INFO: Resource is stopped

May 19 14:27:20 node2 ResourceManager[6862]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /mnt/mysql ext3 start

May 19 14:27:21 node2 Filesystem[7003]: INFO: Running start for /dev/drbd0 on /mnt/mysql

May 19 14:27:21 node2 kernel: kjournald starting. Commit interval 5 seconds

May 19 14:27:21 node2 kernel: EXT3 FS on drbd0, internal journal

May 19 14:27:21 node2 kernel: EXT3-fs: mounted filesystem with ordered data mode.

May 19 14:27:21 node2 Filesystem[6991]: INFO: Success

May 19 14:27:22 node2 ResourceManager[6862]: info: Running /etc/ha.d/resource.d/mysqld start

May 19 14:27:27 node2 ResourceManager[6862]: info: Running /etc/ha.d/resource.d/httpd start

May 19 14:27:28 node2 IPaddr[7432]: INFO: Resource is stopped

May 19 14:27:28 node2 ResourceManager[6862]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.2.100/24/eth0/192.168.2.255 start

May 19 14:27:29 node2 IPaddr[7537]: INFO: Using calculated netmask for 192.168.2.100: 255.255.255.0

May 19 14:27:29 node2 IPaddr[7537]: INFO: eval ifconfig eth0:0 192.168.2.100 netmask 255.255.255.0 broadcast 192.168.2.255

May 19 14:27:29 node2 avahi-daemon[4134]: Registering new address record for 192.168.2.100 on eth0.

May 19 14:27:30 node2 IPaddr[7508]: INFO: Success

May 19 14:27:30 node2 heartbeat: [6849]: info: all HA resource acquisition completed (standby).

May 19 14:27:30 node2 heartbeat: [3954]: info: Standby resource acquisition done [all].

May 19 14:27:30 node2 heartbeat: [3954]: info: remote resource transition completed.

从节点升级为主节点：

查看虚拟ip

启动的端口：

[root@node2 ~]# netstat -tupln

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 3543/portmap

tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 3104/cupsd

tcp 0 0 0.0.0.0:792 0.0.0.0:* LISTEN 3578/rpc.statd

tcp 0 0 :::3306 :::* LISTEN 9013/mysqld

tcp 0 0 :::80 :::* LISTEN 9070/httpd

tcp 0 0 :::22 :::* LISTEN 3826/sshd

udp 0 0 0.0.0.0:786 0.0.0.0:* 3578/rpc.statd

udp 0 0 0.0.0.0:789 0.0.0.0:* 3578/rpc.statd

udp 0 0 0.0.0.0:694 0.0.0.0:* 3975/heartbeat: wri

udp 0 0 0.0.0.0:43491 0.0.0.0:* 3975/heartbeat: wri

udp 0 0 0.0.0.0:5353 0.0.0.0:* 4134/avahi-daemon:

udp 0 0 0.0.0.0:39405 0.0.0.0:* 4134/avahi-daemon:

udp 0 0 0.0.0.0:111 0.0.0.0:* 3543/portmap

udp 0 0 0.0.0.0:631 0.0.0.0:* 3104/cupsd

udp 0 0 :::34911 :::* 4134/avahi-daemon:

udp 0 0 :::5353 :::* 4134/avahi-daemon:

You have new mail in /var/spool/mail/root

磁盘挂载：

 

在磁盘上创建一个文件

echo "haha" > /mnt/mysql/haha.doc

外网访问的结果：

 

 

当主节点恢复正常时，资源又重新回到主节点。

查看在节点node2.ws.cn上创建的文件

 

由上可以看出数据已同步

这样heartbeat+drbd+mysql的高可用群集搭建完成