DNS+KEEPALIVED+LVS集群负载均衡

一、网络结构:

1、两台LVS服务器分别为lvs01、lvs02,IP地址分别为172.18.101.2/25、172.18.101.3/25;两台LVS服务器需要分别安装ipvsadm和keepalived软件,使用ipvsadm来动态调度实际服务器,keepalived用来侦测两台LVS服务器之间通讯,当主DR出现问题,备用自动转换为主DR,响应VIP收到的广播包,并转发到实际服务器上。

2、两台DNS服务器分别为dns01、dns02,IP地址分别为172.18.101.4/25、172.18.101.5/25,VIP=172.18.101.6

以上设备网关在同一台路由器,VIP可以同网段也可以不同网段。使用VIP可以让用户不知道实际DNS的情况下进行解析,同时DNS服务器可以增加到2台以上做一个更大的群组,提高可靠性。

二、具体配置操作

1、LVS服务器,安装ipvsadm和keepalived软件,具体安装步骤可以参考http://www.rootop.org/pages/2102.html,关闭SELinux服务

2、主LVS服务器主要配置文件keepalived.conf:

[root@lvs-ds01 ~]# more/etc/keepalived/keepalived.conf

! Configuration File forkeepalived

 

global_defs {

   notification_email {

     [email protected]

     [email protected]

     [email protected]

   }

   notification_email_from [email protected]

 # smtp_server 183.232.103.164

 # smtp_connect_timeout 30

   router_id lvs_master  (这个id标识名称可以不一样)

}

 

vrrp_instance VI_DNS{

    state MASTER     (参数)

    interface eth0

    virtual_router_id 51  (虚拟路由器id号要主备一致)

    priority 100        (主的优先级要高于备)

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass PASSWORD

    }

    virtual_ipaddress {

        172.18.101.6 dev eth0 label eth0:0

    }

}

 

virtual_server 172.18.101.653 {

    delay_loop 6

    lb_algo rr

    lb_kind DR      (注这里有三种方式:NAT/DR/TUN

    #nat_mask 255.255.255.128

 # persistence_timeout 50

    protocolUDP    (注这里协议选择UDP

  

    real_server 172.18.101.4 53 {

           weight 100              

           MISC_CHECK{

#            connect_timeout 8

#            nb_get_retry 3

#            delay_before_retry 3

#            connect_port 53

             misc_path"/etc/keepalived/udp_check53.sh 172.18.101.4 53"    (由于没有UDP检测,只能自己写脚本检测实际DNS服务器的UDP 53端口)

             misc_timeout 8

        }

    }

 

    real_server 172.18.101.5 53 {

           weight 100    

           MISC_CHECK{

#            connect_timeout 8

#            nb_get_retry 3

#            delay_before_retry 3

#            connect_port 53

             misc_path"/etc/keepalived/udp_check53.sh 172.18.101.5 53"

             misc_timeout 8

        }

    }

}

检测UDP的53端口脚本文件配置udp_check53.sh:

[root@lvs-ds01 ~]# more/etc/keepalived/udp_check53.sh

#!/bin/bash

nc -uz -w 1 $1 $2 | grepsucceeded >/dev/null

exit $?

 

该检测脚本利用命令nc来检查,当检测UDP53端口成功时,会返回一个”succeeded”字段,如下:

[root@lvs-ds01 ~]# nc -uz -w1 172.18.101.4 53

Connection to 172.18.101.453 port [udp/domain] succeeded!

 

Centos7以上版本脚本使用:

手动测试:nc -uz -w 1 172.18.101.4 53

脚本内容:nc -uz -w 1 $1 $2 /dev/null

如果成功,就返回success字段


本实例中在7以下版本,用第一个脚本内容,利用grep “succeeded”来判断是否检测成功。

脚本新建完毕之后,使用chmod +x /etc/keepalived/udp_chekc53.sh来授权操作,否则无法监控。

 

3、备LVS服务器的keepalived.conf文件中,将MASTER参数改为BACKUP,priority值改为90(小于主LVS即可)其他配置不变。

4、查看LVS服务器防火墙状态,没有增加53端口的策略则需要增加:

使用如下命令:

iptables -I INPUT 4 -p tcp--dport 53 -j ACCEPT

iptables -I INPUT 4 -p udp--dport 53 -j ACCEPT

效果如下图所示

 

使用serviceiptables save来保存添加的规则!

 

5、将keepalived服务添加开启启动

chkconfig keepalived on

查看启动项目情况:

6、查看LVS服务器当前网卡情况和调度情况:

[root@lvs-ds01 ~]# ipvsadm

IP Virtual Server version1.2.1 (size=4096)

Prot LocalAddress:PortScheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

UDP  172.18.101.6:domain rr

  -> 172.18.101.4:domain          Route   100   0          0        

  -> 172.18.101.5:domain          Route   100   0          0        

我们发现主服务器上已经有172.18.101.6这个虚拟地址,此时再查看备用服务器上的网卡配置,会发现当主服务器正常的情况下,备用服务器上不会开启VIP:

7、查看两台LVS的日志情况:

主LVS:

 

 

 

 

 

 

备LVS:

8、DNS服务器安装:

1:service iptables stop  //关闭防火墙

2:chkconfig iptables off //开机禁止防火墙启用

3:vi /etc/selinux/config 后修改selinux=disabled//禁用selinux服务

4:yum installbind-chroot.x86_64

5:rndc-confgen -r/dev/urandom -a

6:service named start

7:chkconfig named on

8:配置named.conf

9:测试

 

9、几个重要文件的配置:

Named.conf:(named.conf: /etc/      这个实际是被映射到 /var/named/chroot/etc/   即新建了一个软链接)

[root@real-dns01 ~]# more/etc/named.conf

acl "trusted" {

172.18.0.0/16;

127.0.0.1/32;

10.0.0.0/8;

 

};

 

 

options {

        listen-on port 53 { any; };

        listen-on-v6 port 53 { ::1; };

        directory       "/var/named";

        recursive-clients 100000;

        tcp-clients 100000;

        recursion yes;

        forwarders {202.99.192.66;};

        max-ncache-ttl 10800;

        dump-file      "/var/named/data/cache_dump.db";

        statistics-file"/var/named/data/named_stats.txt";

        memstatistics-file"/var/named/data/named_mem_stats.txt";

 

        // Those options should be usedcarefully because they disable port

        // randomization

         query-source    port 53;

         query-source-v6 port 53;

        allow-recursion {

#               any;

                 trusted;

        };

        allow-transfer {

#               any;

                 trusted;

                };

        allow-query {

                 trusted;

#               any;

                };

};

 

 

#logging {

#        channel default_debug {

#                file"data/named.run";

#                severity dynamic;

#        };

#};

 

logging {

        channel query_log {

               file"/var/log/namequery.log"  versions 3 size 20m;

               severity        info;

               print-time        yes;

               print-category  yes;

              };

              category queries {

                      query_log;

               };

        channel update_log {

               file"/var/log/nameupdate.log" versions 3 size 5m;

               severity        info;

               print-time        yes;

               print-category  yes;

              };

              category update {

                      update_log;

               };

        channel general_log {

                file"/var/log/namegeneral.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category general {

                      general_log;

               };

        channel notify_log {

                file"/var/log/namenotify.log"        versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category notify {

                      notify_log;

               };

        channel lame-servers_log {

                file"/var/log/namelame-servers.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category lame-servers {

                      lame-servers_log;

               };

        channel dnssec_log {

                file"/var/log/namednssec.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category dnssec {

                      dnssec_log;

               };

        channel config_log {

                file"/var/log/nameconfig.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category config {

                      config_log;

               };

        channel network_log {

                file"/var/log/namenetwork.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category network {

                      network_log;

               };

        channel client_log {

                file"/var/log/nameclient.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category client {

                      client_log;

               };

        channel database_log {

                file"/var/log/namedatabase.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category database {

                      database_log;

               };

        channel xfer-in_log {

                file"/var/log/namexfer-in.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category xfer-in {

                      xfer-in_log;

               };

        channel xfer-out_log {

                file"/var/log/namexfer-out.log"       versions 3 size 5m;

                severity        info;

                print-time        yes;

                print-category  yes;

              };

              category xfer-out {

                     xfer-out_log;

               };

};

 

 

 

 

 

 key "rndckey" {

       algorithm hmac-md5;

       secret"X4hY8z7oQNldOCn9L5yrUQ==";

 };

 

 controls {

       inet 127.0.0.1 port 953

               allow { 127.0.0.1; } keys {"rndckey"; };

 };

include"/var/named/data/forwarders.conf";

# End of named.conf

 

Forwarders.conf:

[root@real-dns01 ~]# more/etc/forwarders.conf  (软链接

 

zone "baidu.com" {

       type forward;

       forward first;

       forwarders {120.196.165.24;};

};

 

zone "shifen.com"{

       type forward;

       forward first;

       forwarders {120.196.165.24;};

};

 

 

 

10、DNS上的脚本配置:

[root@real-dns01 ~]# more/etc/init.d/dns_rs.ctl

#!/bin/bash

#chkconfig:345 85 15   (加入开机启动,可使用chkconfig检查一下

. /etc/init.d/functions

 VIP=172.18.101.6

 

case "$1" in

start)

    echo "start LVS of Realserver DRmode"

    /sbin/ifconfig lo:0 ${VIP} netmask255.255.255.255 broadcast 172.18.101.6 up

    route add -host ${VIP} dev lo:0

 echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore

 echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce

 echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore

 echo "2">/proc/sys/net/ipv4/conf/all/arp_announce

    rndc reload

   ;;

stop)

  echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

  echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce

  echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore

  echo "0">/proc/sys/net/ipv4/conf/all/arp_announce

    echo "stop LVS of Realserver DRmode"

    /sbin/ifconfig lo:0 down

    route del -host ${VIP} dev lo:0

   ;;

*)

   echo "Usage: $0 {start|stop}"

   exit 1

  Esac

脚本同样要赋予权限:chmod +x/etc/init.d/dns_rs.ctl

手动执行:/etc/init.d/dns_rs.ctlstart 提示脚本中的内容信息“start LVS of Realserver DR mode”

 

11、DNS网卡配置查看,

dns01:

Dns02:

 

在服务器上使用netstat-anu查看udp端口是否侦听

四、原理分析

1、http://www.linuxidc.com/Linux/2012-05/60950.htm

2、http://pengai.blog.51cto.com/6326789/1706632

3、每台DNS上的脚本,通过设置arp_ignore和arp_announce的值来进行限制和响应,当用户访问VIP时,DNS上的VIP不响应广播包,此时只由LVS主机来响应,再将请求发给实际DNS服务器,DNS服务器在将结果直接返回给用户

 

arp_ignore=1,系统只回答目的IP为是本地IP的包。也就是对广播包不做响应。

arp_announce=2,系统忽略IP包的源地址(source address),而根据目标主机(target host),选择本地地址。

 

而且凡是能收到对VIPARP广播报文的网口,都需要设置。设置的方法是修改/etc/sysctl.conf文件

 

net.ipv4.ip_forward = 1

net.ipv4.conf.lo.arp_ignore= 1

net.ipv4.conf.lo.arp_announce= 2

net.ipv4.conf.all.arp_ignore= 1

net.ipv4.conf.all.arp_announce= 2

另外,解决sysctl -p 出现“xxx unknow key”的问题,使用命令:

modprobe bridge来解决

 

五、参考文档

1、http://blog.csdn.net/yanziguishi/article/details/6743504

2、http://blog.csdn.net/kumu_linux/article/details/8739089

3、http://www.rootop.org/pages/2102.html

4、http://nosmoking.blog.51cto.com/3263888/1616037

5、http://pengai.blog.51cto.com/6326789/1706632

6、http://blog.csdn.net/py_shell/article/details/53067847?locationNum=7&fps=1

7、http://www.linuxidc.com/Linux/2012-05/60950.htm

8、http://mikeluwen.blog.51cto.com/5619187/1390215

9、http://blog.163.com/passc_lee/blog/static/2152541462013615112019346/

10、http://www.cnblogs.com/xiaocen/p/3709869.html

 


你可能感兴趣的:(服务器平台部署文档)