lvs+keepalived+mysql cluster实现负载均衡
一：前言
Mysql-cluster本身实现了高可用，但是其sql节点并没有实现负载均衡。怎样让程序能均衡地访问sql节点呢，程序里写哪个数据库服务器IP呢？
LVS是实现负载均衡作用的，即将客户端的需求采用特定的负载均衡算法分发到后端的Web应用服务器上。
Keepalived是用来实现高可用的，提供健康检查，故障转移，即保证主LVS宕机后，从LVS可以在很短时间顶上，从而保证了系统或网站的稳定性。Keepalived在后端的realserver接收不到主节点的信息之后，keepalived能够自己调用ipvsadm命令生成规则，能够自动实现，将主节点的VIP以及ipvs规则“拿过来”，应用在从节点上，继续为用户服务。
程序里写虚拟IP即可。
拓扑图：
二：实验环境
--已经安装好了mysql cluster
Mysql cluster安装,请参考：
http://blog.csdn.net/yabingshi_tech/article/details/47952135
三：实验步骤
3.1：安装ipvsadm
分别在lvs-master(主调度器）和lvs-backup(从调度器）上安装.
#下载ipvsadm
点击这里下载ipvsadm:
http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
#查看内核版本
uname -r
2.6.32-431.el6.x86_64
先进cd /usr/src/kernels/看有没有该文件：
2.6.32-431.el6.x86_64
如果没有，需要：yum -y install kernel-devel
否则，在后面编译的时候回报错：如libipvs.h:14:23: error: net/ip_vs.h: No such file or directory
#创建软连接
ln -s /usr/src/kernels/2.6.32-573.3.1.el6.x86_64/ /usr/src/linux
注意：软连接时，需要提供2.6.32-573.3.1.el6.x86_64的绝对路径，如果直接进入/usr/src/kernels目录，这样软连接是不行的：ln -s 2.6.32-573.3.1.el6.x86_64/ /usr/src/linux
#编译
cd /download
tar -xvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
[root@ser6-70 ipvsadm-1.24]# make
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o ipvsadm.o ipvsadm.c
ipvsadm.c: In function ‘print_largenum’:
ipvsadm.c:1564: warning: field width should have type ‘int’, but argument 2 has type ‘size_t’
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o config_stream.o config_stream.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o dynamic_array.o dynamic_array.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -o ipvsadm ipvsadm.o config_stream.o dynamic_array.o libipvs/libipvs.a [root@ser6-70 ipvsadm-1.24]# make install
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
if [ ! -d /sbin ]; then mkdir -p /sbin; fi
install -m 0755 -s ipvsadm /sbin
install -m 0755 ipvsadm-save /sbin
install -m 0755 ipvsadm-restore /sbin
[ -d /usr/man/man8 ] || mkdir -p /usr/man/man8
install -m 0644 ipvsadm.8 /usr/man/man8
install -m 0644 ipvsadm-save.8 /usr/man/man8
install -m 0644 ipvsadm-restore.8 /usr/man/man8
if [ -d /etc/rc.d/init.d ]; then \
install -m 0755 ipvsadm.sh /etc/rc.d/init.d/ipvsadm; \
fi#验证是否安装成功
[root@ser6-70 ipvsadm-1.24]# /sbin/ipvsadm -v
ipvsadm v1.24 2005/12/10 (compiled with getopt_long and IPVS v1.2.1)
说明安装成功。
3.2：安装keepalived
分别在lvs_master(主调度器）和lvs-backup(从调度器）上安装。
安装keepalived前先要安装popt-devel，openssl :
yum install popt-devel -y
yum install -y openssl openssl-devel
#安装keepalived
wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
#解压
tar zxvf keepalived-1.1.19.tar.gz
[root@ser6-70 linux]# cd keepalived-1.1.19
[root@ser6-70 keepalived-1.1.19]# ./configure --prefix=/usr/local/keepalived
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for strip... strip
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for unistd.h... (cached) yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/md5.h usability... yes
checking openssl/md5.h presence... yes
checking for openssl/md5.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking for MD5_Init in -lcrypto... yes
checking for SSL_CTX_new in -lssl... yes
checking for poptGetContext in -lpopt... yes
checking for kernel version... 2.6.32
checking net/ip_vs.h usability... yes
checking net/ip_vs.h presence... yes
checking for net/ip_vs.h... yes
checking for IPVS syncd support... yes
checking for an ANSI C-conforming const... yes
checking for pid_t... yes
checking whether time.h and sys/time.h may both be included... yes
checking whether gcc needs -traditional... no
checking for working memcmp... yes
checking return type of signal handlers... void
checking for gettimeofday... yes
checking for select... yes
checking for socket... yes
checking for strerror... yes
checking for strtol... yes
checking for uname... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating genhash/Makefile
config.status: WARNING: 'genhash/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating keepalived/core/Makefile
config.status: creating keepalived/include/config.h
config.status: creating keepalived.spec
config.status: creating keepalived/Makefile
config.status: WARNING: 'keepalived/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating lib/Makefile
config.status: creating keepalived/vrrp/Makefile
config.status: creating keepalived/check/Makefile
config.status: creating keepalived/libipvs-2.6/Makefile
Keepalived configuration
------------------------
Keepalived version : 1.1.19
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
Use Debug flags : No以下全部为 YES, 表示配置成功。
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
#编译
[root@ser6-70 keepalived-1.1.19]# make
make -C lib || exit 1;
make[1]: Entering directory `/download/keepalived-1.1.19/lib'
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c memory.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c utils.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c notify.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c timer.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c scheduler.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c vector.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c list.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c html.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c parser.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c signals.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c logger.c
make[1]: Leaving directory `/download/keepalived-1.1.19/lib'
make -C keepalived
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/core'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c main.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c pidfile.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c layer4.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c smtp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_parser.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/core'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/check'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_parser.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_api.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_tcp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_http.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_ssl.c
check_ssl.c: In function ‘build_ssl_ctx’:
check_ssl.c:79: warning: assignment discards qualifiers from pointer target type
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_smtp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_misc.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipwrapper.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipvswrapper.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/check'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/vrrp'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_parser.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_notify.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_scheduler.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_sync.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_index.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_netlink.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_arp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_if.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_track.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipaddress.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_iproute.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipsecah.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/vrrp'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
gcc -g -O2 -I/usr/src/linux/include -Wall -Wunused -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
rm libipvs.o
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
Building ../bin/keepalived
strip ../bin/keepalived
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o main.o main.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o sock.o sock.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o layer4.o layer4.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o http.o http.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o ssl.o ssl.c
ssl.c: In function ‘init_ssl’:
ssl.c:50: warning: assignment discards qualifiers from pointer target type
Building ../bin/genhash
strip ../bin/genhash
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
Make complete[root@ser6-70 keepalived-1.1.19]# make install
make -C keepalived install
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
install -d /usr/local/keepalived/sbin
install -m 700 ../bin/keepalived /usr/local/keepalived/sbin/
install -d /usr/local/keepalived/etc/rc.d/init.d
install -m 755 etc/init.d/keepalived.init /usr/local/keepalived/etc/rc.d/init.d/keepalived
install -d /usr/local/keepalived/etc/sysconfig
install -m 755 etc/init.d/keepalived.sysconfig /usr/local/keepalived/etc/sysconfig/keepalived
install -d /usr/local/keepalived/etc/keepalived/samples
install -m 644 etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/
install -m 644 ../doc/samples/* /usr/local/keepalived/etc/keepalived/samples/
install -d /usr/local/keepalived/share/man/man5
install -d /usr/local/keepalived/share/man/man8
install -m 644 ../doc/man/man5/keepalived.conf.5 /usr/local/keepalived/share/man/man5
install -m 644 ../doc/man/man8/keepalived.8 /usr/local/keepalived/share/man/man8
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash install
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
install -d /usr/local/keepalived/bin
install -m 755 ../bin/genhash /usr/local/keepalived/bin/
install -d /usr/local/keepalived/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
[root@ser6-70 keepalived-1.1.19]#
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived3.3：配置keepadlived
3.3.1 lvs-master配置
vi /etc/keepalived/keepalived.conf
添加内容：
global_defs {
notification_email {
[email protected] #email 通知
}
notification_email_from [email protected]
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS1 # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_sync_group test { #设置vrrp组
group {
loadbalance
}
}
vrrp_instance loadbalance {
state MASTER #设置lvs的状态，报错MASTER和BACKUP两种，必须大写
interface eth0 #设置对外服务的接口
lvs_sync_daemon_inteface eth0 #设置lvs监听的接口
virtual_router_id 51 #设置虚拟路由表示
priority 180 #设置优先级，数值越大，优先级越高
advert_int 5 #设置同步时间间隔
authentication { #设置验证类型和密码
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #设置lvs vip
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6 #健康检查时间间隔
lb_algo rr #负载均衡调度算法
lb_kind DR #负载均衡转发规则
#persistence_timeout 20 #设置会话保持时间，对bbs等很有用
protocol TCP #协议
real_server 192.168.6.66 3306 {
weight 3 #设置权重
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}3.3.2 lvs-backup 配置
vi /etc/keepalived/keepalived.conf
添加：
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS2
}
vrrp_sync_group test {
group {
loadbalance
}
}
vrrp_instance loadbalance {
state BACKUP
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 51
priority 150
advert_int 5
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6
lb_algo rr
lb_kind DR
#persistence_timeout 20
protocol TCP
real_server 192.168.6.66 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}3.3.3 Realserver配置
在所有sql节点上配置：
vi /etc/rc.d/init.d/realserver.sh
添加：
#!/bin/bash
# description: Config realserver lo and apply noarp
SNS_VIP=192.168.6.201
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0# 赋予执行权限
chmod +x /etc/rc.d/init.d/functions
chmod +x /etc/rc.d/init.d/realserver.sh
#启动
/etc/rc.d/init.d/realserver.sh start
启动后使用ifconfig命令能看到类似下面新增的：
lo:0 Link encap:Local Loopback
inet addr:192.168.6.202 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
注意：在sql节点一定要启动这个，否则后面无法连接mysql虚拟ip：ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.6.202' (110)
3.3.4：启动keepalived
在lvs-master,lvs-backup上启动：
service keepalived start
Starting keepalived: [ OK ]
启动keepalived后，虚拟ip就能ping通了。
#查看状态
在lvs-master查看：
[root@ser6-70 keepalived]# ip addr
1：lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever 可以看到虚拟 IP.
[root@ser6-70 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.6.202:mysql rr
-> 192.168.6.67:mysql Route 3 2 0
-> 192.168.6.66:mysql Route 3 0 0 -- 在 lvs-backup 上查看状态
[root@ser6-71 keepalived]# ip addr
1：lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever 这里没看到虚拟 ip ，说明这个是从 lvs.
[root@ser6-71 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.6.202:mysql rr
-> 192.168.6.67:mysql Route 3 0 0
-> 192.168.6.66:mysql Route 3 0 0 3.4 建允许远程访问的用户
在sql节点上创建允许远程访问的用户：
grant select,update,delete,insert on *.* to 'dandan' identified by 'dandan';
3.5 测试
3.5.1 测试lvs负载均衡
#在客户端192.168.6.51连接虚拟ip
[root@ser6-51 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3677
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| dba |
| mysql |
| ndbinfo |
| performance_schema |
| test |
+--------------------+
6 rows in set (0.00 sec)
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+---------------+
| Tables_in_dba |
+---------------+
| t2 |
+---------------+
1 row in set (0.01 sec)
mysql> select * from t2;
+------+
| id |
+------+
| 1 |
| 4 |
| 2 |
| 3 |
| 5 |
| 1 |
+------+
6 rows in set (0.01 sec)
mysql> delete from t2 where id = 1;
Query OK, 2 rows affected (0.07 sec)
mysql> select * from t2;
+------+
| id |
+------+
| 3 |
| 5 |
| 2 |
| 4 |
+------+
4 rows in set (0.00 sec)
mysql> 在 lvs-master 上查看：
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:29 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
看到192.168.6.51通过虚拟ip，连接到了192.168.6.66.
进sql节点192.168.6.66，看是否真的删除了数据。
[root@ser6-66 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3731
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from t2;
+------+
| id |
+------+
| 3 |
| 5 |
| 2 |
| 4 |
+------+
4 rows in set (0.02 sec)真的删除了，说明客户端成功通过虚拟 ip 连接到了 sql 节点。
#在客户端192.168.6.52上登录vip
[root@ser6-52 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3739
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> 在 lvs-master 上查看状态：
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 12:36 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
TCP 14:54 ESTABLISHED 192.168.6.52:52348 192.168.6.202:3306 192.168.6.67:3306
显示192.168.6.52连接到了192.168.6.67.说明lvs实现了负载均衡。两个客户端分别连上了不同的sql节点。
注意：只能在lvs-master上输入ipvsadm -lcn进行查看，在lvs-backup上查看是空的：
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
3.5.2 测试keepalived高可用
现在把lvs-master的keepalived关闭，看是否能实现高可用。
[root@ser6-70 keepalived]# service keepalived stop
Stopping keepalived: [ OK ]
过几秒钟后，在lvs-backup上进行查看：
[root@ser6-71 keepalived]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever 看到虚拟 IP 已经漂移到了 lvs-backup.
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:25 ESTABLISHED 192.168.6.52:54311 192.168.6.202:3306 192.168.6.67:3306
TCP 14:49 ESTABLISHED 192.168.6.51:29604 192.168.6.202:3306 192.168.6.66:3306
客户端也能够正常访问vip节点，这里不再赘述。
在lvs-master上启动keepalived后，vip会自动漂移到master节点上。
[root@ser6-70 keepalived]# service keepalived start
Starting keepalived: [ OK ]
[root@ser6-70 keepalived]# ip addr
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever --本篇文章主要参考：
http://developer.51cto.com/art/201405/440955_all.htm
http://7567567.blog.51cto.com/706378/578289/