lvs+keepalived+mysql cluster实现负载均衡
一：前言
Mysql-cluster本身实现了高可用，但是其sql节点并没有实现负载均衡。怎样让程序能均衡地访问sql节点呢，程序里写哪个数据库服务器IP呢？
LVS是实现负载均衡作用的，即将客户端的需求采用特定的负载均衡算法分发到后端的Web应用服务器上。
Keepalived是用来实现高可用的，提供健康检查，故障转移，即保证主LVS宕机后，从LVS可以在很短时间顶上，从而保证了系统或网站的稳定性。Keepalived在后端的realserver接收不到主节点的信息之后，keepalived能够自己调用ipvsadm命令生成规则，能够自动实现，将主节点的VIP以及ipvs规则“拿过来”，应用在从节点上，继续为用户服务。
程序里写虚拟IP即可。
拓扑图：
二：实验环境
--已经安装好了mysql cluster
Mysql cluster安装,请参考：
http://blog.csdn.net/yabingshi_tech/article/details/47952135
三：实验步骤
3.1：安装ipvsadm
分别在lvs-master(主调度器）和lvs-backup(从调度器）上安装.
#下载ipvsadm
点击这里下载ipvsadm:
http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
#查看内核版本
uname -r
2.6.32-431.el6.x86_64
先进cd /usr/src/kernels/看有没有该文件：
2.6.32-431.el6.x86_64
如果没有，需要：yum -y install kernel-devel
否则，在后面编译的时候回报错：如libipvs.h:14:23: error: net/ip_vs.h: No such file or directory
#创建软连接
ln -s /usr/src/kernels/2.6.32-573.3.1.el6.x86_64/ /usr/src/linux
注意：软连接时，需要提供2.6.32-573.3.1.el6.x86_64的绝对路径，如果直接进入/usr/src/kernels目录，这样软连接是不行的：ln -s 2.6.32-573.3.1.el6.x86_64/ /usr/src/linux
#编译
cd /download tar -xvf ipvsadm-1.24.tar.gz cd ipvsadm-1.24 [root@ser6-70 ipvsadm-1.24]# make make -C libipvs make[1]: Entering directory `/download/ipvsadm-1.24/libipvs' gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.c ar rv libipvs.a libipvs.o ar: creating libipvs.a a - libipvs.o make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs' gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o ipvsadm.o ipvsadm.c ipvsadm.c: In function ‘print_largenum’: ipvsadm.c:1564: warning: field width should have type ‘int’, but argument 2 has type ‘size_t’ gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o config_stream.o config_stream.c gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o dynamic_array.o dynamic_array.c gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -o ipvsadm ipvsadm.o config_stream.o dynamic_array.o libipvs/libipvs.a
[root@ser6-70 ipvsadm-1.24]# make install make -C libipvs make[1]: Entering directory `/download/ipvsadm-1.24/libipvs' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs' if [ ! -d /sbin ]; then mkdir -p /sbin; fi install -m 0755 -s ipvsadm /sbin install -m 0755 ipvsadm-save /sbin install -m 0755 ipvsadm-restore /sbin [ -d /usr/man/man8 ] || mkdir -p /usr/man/man8 install -m 0644 ipvsadm.8 /usr/man/man8 install -m 0644 ipvsadm-save.8 /usr/man/man8 install -m 0644 ipvsadm-restore.8 /usr/man/man8 if [ -d /etc/rc.d/init.d ]; then \ install -m 0755 ipvsadm.sh /etc/rc.d/init.d/ipvsadm; \ fi
#验证是否安装成功
[root@ser6-70 ipvsadm-1.24]# /sbin/ipvsadm -v
ipvsadm v1.24 2005/12/10 (compiled with getopt_long and IPVS v1.2.1)
说明安装成功。
3.2：安装keepalived
分别在lvs_master(主调度器）和lvs-backup(从调度器）上安装。
安装keepalived前先要安装popt-devel，openssl :
yum install popt-devel -y
yum install -y openssl openssl-devel
#安装keepalived
wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
#解压
tar zxvf keepalived-1.1.19.tar.gz
[root@ser6-70 linux]# cd keepalived-1.1.19
[root@ser6-70 keepalived-1.1.19]# ./configure --prefix=/usr/local/keepalived checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for a BSD-compatible install... /usr/bin/install -c checking for strip... strip checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/wait.h that is POSIX.1 compatible... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking syslog.h usability... yes checking syslog.h presence... yes checking for syslog.h... yes checking for unistd.h... (cached) yes checking sys/ioctl.h usability... yes checking sys/ioctl.h presence... yes checking for sys/ioctl.h... yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking openssl/ssl.h usability... yes checking openssl/ssl.h presence... yes checking for openssl/ssl.h... yes checking openssl/md5.h usability... yes checking openssl/md5.h presence... yes checking for openssl/md5.h... yes checking openssl/err.h usability... yes checking openssl/err.h presence... yes checking for openssl/err.h... yes checking for MD5_Init in -lcrypto... yes checking for SSL_CTX_new in -lssl... yes checking for poptGetContext in -lpopt... yes checking for kernel version... 2.6.32 checking net/ip_vs.h usability... yes checking net/ip_vs.h presence... yes checking for net/ip_vs.h... yes checking for IPVS syncd support... yes checking for an ANSI C-conforming const... yes checking for pid_t... yes checking whether time.h and sys/time.h may both be included... yes checking whether gcc needs -traditional... no checking for working memcmp... yes checking return type of signal handlers... void checking for gettimeofday... yes checking for select... yes checking for socket... yes checking for strerror... yes checking for strtol... yes checking for uname... yes configure: creating ./config.status config.status: creating Makefile config.status: creating genhash/Makefile config.status: WARNING: 'genhash/Makefile.in' seems to ignore the --datarootdir setting config.status: creating keepalived/core/Makefile config.status: creating keepalived/include/config.h config.status: creating keepalived.spec config.status: creating keepalived/Makefile config.status: WARNING: 'keepalived/Makefile.in' seems to ignore the --datarootdir setting config.status: creating lib/Makefile config.status: creating keepalived/vrrp/Makefile config.status: creating keepalived/check/Makefile config.status: creating keepalived/libipvs-2.6/Makefile Keepalived configuration ------------------------ Keepalived version : 1.1.19 Compiler : gcc Compiler flags : -g -O2 Extra Lib : -lpopt -lssl -lcrypto Use IPVS Framework : Yes IPVS sync daemon support : Yes Use VRRP Framework : Yes Use Debug flags : No
以下全部为 YES, 表示配置成功。
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
#编译
[root@ser6-70 keepalived-1.1.19]# make make -C lib || exit 1; make[1]: Entering directory `/download/keepalived-1.1.19/lib' gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c memory.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c utils.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c notify.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c timer.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c scheduler.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c vector.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c list.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c html.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c parser.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c signals.c gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c logger.c make[1]: Leaving directory `/download/keepalived-1.1.19/lib' make -C keepalived make[1]: Entering directory `/download/keepalived-1.1.19/keepalived' make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/core' gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c main.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c daemon.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c pidfile.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c layer4.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c smtp.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_data.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_parser.c make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/core' make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/check' gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_daemon.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_data.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_parser.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_api.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_tcp.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_http.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_ssl.c check_ssl.c: In function ‘build_ssl_ctx’: check_ssl.c:79: warning: assignment discards qualifiers from pointer target type gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_smtp.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_misc.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipwrapper.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipvswrapper.c make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/check' make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/vrrp' gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_daemon.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_data.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_parser.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_notify.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_scheduler.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_sync.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_index.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_netlink.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_arp.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_if.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_track.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipaddress.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_iproute.c gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipsecah.c make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/vrrp' make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6' gcc -g -O2 -I/usr/src/linux/include -Wall -Wunused -c -o libipvs.o libipvs.c ar rv libipvs.a libipvs.o ar: creating libipvs.a a - libipvs.o rm libipvs.o make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6' Building ../bin/keepalived strip ../bin/keepalived Make complete make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived' make -C genhash make[1]: Entering directory `/download/keepalived-1.1.19/genhash' gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o main.o main.c gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o sock.o sock.c gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o layer4.o layer4.c gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o http.o http.c gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o ssl.o ssl.c ssl.c: In function ‘init_ssl’: ssl.c:50: warning: assignment discards qualifiers from pointer target type Building ../bin/genhash strip ../bin/genhash Make complete make[1]: Leaving directory `/download/keepalived-1.1.19/genhash' Make complete
[root@ser6-70 keepalived-1.1.19]# make install make -C keepalived install make[1]: Entering directory `/download/keepalived-1.1.19/keepalived' install -d /usr/local/keepalived/sbin install -m 700 ../bin/keepalived /usr/local/keepalived/sbin/ install -d /usr/local/keepalived/etc/rc.d/init.d install -m 755 etc/init.d/keepalived.init /usr/local/keepalived/etc/rc.d/init.d/keepalived install -d /usr/local/keepalived/etc/sysconfig install -m 755 etc/init.d/keepalived.sysconfig /usr/local/keepalived/etc/sysconfig/keepalived install -d /usr/local/keepalived/etc/keepalived/samples install -m 644 etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/ install -m 644 ../doc/samples/* /usr/local/keepalived/etc/keepalived/samples/ install -d /usr/local/keepalived/share/man/man5 install -d /usr/local/keepalived/share/man/man8 install -m 644 ../doc/man/man5/keepalived.conf.5 /usr/local/keepalived/share/man/man5 install -m 644 ../doc/man/man8/keepalived.8 /usr/local/keepalived/share/man/man8 make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived' make -C genhash install make[1]: Entering directory `/download/keepalived-1.1.19/genhash' install -d /usr/local/keepalived/bin install -m 755 ../bin/genhash /usr/local/keepalived/bin/ install -d /usr/local/keepalived/share/man/man1 install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1 make[1]: Leaving directory `/download/keepalived-1.1.19/genhash' [root@ser6-70 keepalived-1.1.19]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ mkdir /etc/keepalived
3.3：配置keepadlived
3.3.1 lvs-master配置
vi /etc/keepalived/keepalived.conf
添加内容：
global_defs {
notification_email {
[email protected] #email 通知
}
notification_email_from [email protected]
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS1 # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_sync_group test { #设置vrrp组
group {
loadbalance
}
}
vrrp_instance loadbalance {
state MASTER #设置lvs的状态，报错MASTER和BACKUP两种，必须大写
interface eth0 #设置对外服务的接口
lvs_sync_daemon_inteface eth0 #设置lvs监听的接口
virtual_router_id 51 #设置虚拟路由表示
priority 180 #设置优先级，数值越大，优先级越高
advert_int 5 #设置同步时间间隔
authentication { #设置验证类型和密码
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { #设置lvs vip
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6 #健康检查时间间隔
lb_algo rr #负载均衡调度算法
lb_kind DR #负载均衡转发规则
#persistence_timeout 20 #设置会话保持时间，对bbs等很有用
protocol TCP #协议
real_server 192.168.6.66 3306 {
weight 3 #设置权重
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
3.3.2 lvs-backup 配置
vi /etc/keepalived/keepalived.conf
添加：
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS2
}
vrrp_sync_group test {
group {
loadbalance
}
}
vrrp_instance loadbalance {
state BACKUP
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 51
priority 150
advert_int 5
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6
lb_algo rr
lb_kind DR
#persistence_timeout 20
protocol TCP
real_server 192.168.6.66 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
3.3.3 Realserver配置
在所有sql节点上配置：
vi /etc/rc.d/init.d/realserver.sh
添加：
#!/bin/bash
# description: Config realserver lo and apply noarp
SNS_VIP=192.168.6.201
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
# 赋予执行权限
chmod +x /etc/rc.d/init.d/functions
chmod +x /etc/rc.d/init.d/realserver.sh
#启动
/etc/rc.d/init.d/realserver.sh start
启动后使用ifconfig命令能看到类似下面新增的：
lo:0 Link encap:Local Loopback
inet addr:192.168.6.202 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
注意：在sql节点一定要启动这个，否则后面无法连接mysql虚拟ip：ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.6.202' (110)
3.3.4：启动keepalived
在lvs-master,lvs-backup上启动：
service keepalived start
Starting keepalived: [ OK ]
启动keepalived后，虚拟ip就能ping通了。
#查看状态
在lvs-master查看：
[root@ser6-70 keepalived]# ip addr
1：lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever
可以看到虚拟 IP.
[root@ser6-70 keepalived]# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.6.202:mysql rr -> 192.168.6.67:mysql Route 3 2 0 -> 192.168.6.66:mysql Route 3 0 0
-- 在 lvs-backup 上查看状态
[root@ser6-71 keepalived]# ip addr
1：lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever
这里没看到虚拟 ip ，说明这个是从 lvs.
[root@ser6-71 keepalived]# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.6.202:mysql rr -> 192.168.6.67:mysql Route 3 0 0 -> 192.168.6.66:mysql Route 3 0 0
3.4 建允许远程访问的用户
在sql节点上创建允许远程访问的用户：
grant select,update,delete,insert on *.* to 'dandan' identified by 'dandan';
3.5 测试
3.5.1 测试lvs负载均衡
#在客户端192.168.6.51连接虚拟ip
[root@ser6-51 ~]# mysql -u dandan -pdandan -h 192.168.6.202 Warning: Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3677 Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL) Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | dba | | mysql | | ndbinfo | | performance_schema | | test | +--------------------+ 6 rows in set (0.00 sec) mysql> use dba; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +---------------+ | Tables_in_dba | +---------------+ | t2 | +---------------+ 1 row in set (0.01 sec) mysql> select * from t2; +------+ | id | +------+ | 1 | | 4 | | 2 | | 3 | | 5 | | 1 | +------+ 6 rows in set (0.01 sec) mysql> delete from t2 where id = 1; Query OK, 2 rows affected (0.07 sec) mysql> select * from t2; +------+ | id | +------+ | 3 | | 5 | | 2 | | 4 | +------+ 4 rows in set (0.00 sec) mysql>
在 lvs-master 上查看：
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:29 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
看到192.168.6.51通过虚拟ip，连接到了192.168.6.66.
进sql节点192.168.6.66，看是否真的删除了数据。
[root@ser6-66 ~]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3731 Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL) Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use dba; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from t2; +------+ | id | +------+ | 3 | | 5 | | 2 | | 4 | +------+ 4 rows in set (0.02 sec)
真的删除了，说明客户端成功通过虚拟 ip 连接到了 sql 节点。
#在客户端192.168.6.52上登录vip
[root@ser6-52 ~]# mysql -u dandan -pdandan -h 192.168.6.202 Warning: Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3739 Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL) Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
在 lvs-master 上查看状态：
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 12:36 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
TCP 14:54 ESTABLISHED 192.168.6.52:52348 192.168.6.202:3306 192.168.6.67:3306
显示192.168.6.52连接到了192.168.6.67.说明lvs实现了负载均衡。两个客户端分别连上了不同的sql节点。
注意：只能在lvs-master上输入ipvsadm -lcn进行查看，在lvs-backup上查看是空的：
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
3.5.2 测试keepalived高可用
现在把lvs-master的keepalived关闭，看是否能实现高可用。
[root@ser6-70 keepalived]# service keepalived stop
Stopping keepalived: [ OK ]
过几秒钟后，在lvs-backup上进行查看：
[root@ser6-71 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever
看到虚拟 IP 已经漂移到了 lvs-backup.
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:25 ESTABLISHED 192.168.6.52:54311 192.168.6.202:3306 192.168.6.67:3306
TCP 14:49 ESTABLISHED 192.168.6.51:29604 192.168.6.202:3306 192.168.6.66:3306
客户端也能够正常访问vip节点，这里不再赘述。
在lvs-master上启动keepalived后，vip会自动漂移到master节点上。
[root@ser6-70 keepalived]# service keepalived start
Starting keepalived: [ OK ]
[root@ser6-70 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever
--本篇文章主要参考：
http://developer.51cto.com/art/201405/440955_all.htm
http://7567567.blog.51cto.com/706378/578289/